DOCSLIB.ORG

Evaluating Methods of Estimating Common Risk Difference for Stratified Binomial Trials Clinical Research Services

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Evaluating Methods of Estimating Common Risk Difference for Stratified Binomial Trials Clinical Research Services Kate Fisher, MA Biostatistics Global Data Operations Clinical Research Services Evaluating methods of estimating common risk difference for stratified binomial trials for less common events Background Methods Cont. Results Summary Many clinical trials use a stratified randomization approach to Simulation Cont. Example Risk Difference Estimates (pct-ptx) Stratified Score CI • The Score CI has a slightly higher chance of being declared ensure balanced treatment assignment within subgroups, helping The performance of the common risk difference estimation significant compared to the Stratified Newcombe in the Young, Low risk to more accurately estimate treatment effect. For example, a trial methods are investigated under 2 settings: ranges of a reasonably powered study (P(significant)>80%) may stratify the randomization on gender, age groupings, or other Young, High risk 1. Varying the magnitude of risk difference(pct- ptx) • The probability of capturing the true treatment difference baseline covariate that investigators believe may influence (maintaining homogenous risk difference across Old, Low risk was below the 95% value for all methods, but increasing for treatment effect on a binomial outcome. For stratified binomial strata) Old, High risk values between 0.08 and 0.024. trials, a risk difference can be calculated per stratum. These Common • Score(Miettinen-Nurminen) are more likely to capture estimates can be combined to produce a common risk difference 2. Varying the magnitude of heterogeneity of risk smaller differences(<0.08) and Stratified Newcombe CIs to describe the overall treatment effect. However, there are many difference MH Wald-type(Sato) were more likely to capture larger treatment differences. ways to estimate the common risk difference and confidence For 1), the magnitude of the difference between the New(Klingenberg) • Stratified Newcombe CIs were the widest, followed by interval(CI). This research explores confidence intervals using a probabilities of event for control arm(pct) and treatment arm Stratified Newcombe variety of methods: Wald-Type(Sato)[3], Stratified Newcombe[4], (ptx) is varied from 0% to 2.5%, using .05% increments. Score. Score(Miettinen-Nurminen) Score (Miettinen-Nurminen)[5,6], and a new approach presented For 2), heterogeneity is introduced by sampling from a • The MH Wald-type CI and the new(Klingenberg) method -0.05 -0.025 0 0.025 0.05 0.075 by Klingenberg[7]. Cochrane-Mantel-Haenzel[1,2] weights are uniform distribution centered at 0 with bounds varying at performed similarly, with narrow confidence intervals, used when constructing the Wald and Stratified Newcombe 0.01%, 0.1% and 1% to add noise to ptx for each stratum. slightly higher chance of being significant, and worse confidence intervals. For this setting, pct is kept constant at 3% and ptx is 2% + coverage probability. noise. Each simulation is run for 10,000 iterations. 0.0149 Methods • Introducing heterogeneity didn’t appreciably alter the results The following is captured to help determine if certain when average risk difference was 1%. 0.0148 MH Wald-type(Sato) CI methods outperform others across simulations: • Cochrane-Mantel-Haenzel(MH) weights are used to – How often 0, indicating an insignificant result, is in 0.0147 Conclusions combine the stratum risk differences to produce the the CI common risk difference and variance of the estimate 0.0146 • In this particular setting of sample sizes and poverall there – How often the true underlying risk difference is Average CI width are not drastic differences in conclusions between the using the Sato variance estimator. Average CI width captured in the CI (coverage probability) 0.0145 New CI presented by Klingenberg methods, perhaps because of the event probability not – Average width of CI 0.0144 being too rare and the sample sizes being very large and • Similar to Wald-type, but estimates the variance under This simulation was run in R Studio (R version 3.2.0)[9] uneven. the null to construct the CI. Wald(Sato) New(Klingenberg) Stratified Newcombe Score(M-N) • However, these results suggest using the Score CI due to Stratified Newcombe confidence limits Results a good balance between high coverage probability of the • The stratified Wilson confidence limits(CL) for each Probability of significant CIs over varying treatment differences true risk difference, probability of declaring significance, P =2.5% treatment group probability of event are combined using overall ptx-pct=-0.01 low heterogenity med heterogenity high heterogenity and CI width. (no heterogenity) uni[-1e-04,1e-04] uni[-0.001,0.001] uni[-0.01,0.01] 1.0 MH weights to get common upper and lower CLs, which 0.16 MH Wald-type(Sato) Wald(Sato) • An r function has now been written for the Stratified 0.90 0.90 0.90 are used to calculate a common risk difference CL New(Klingenberg) 0.90 New(Klingenberg) 0.8 Stratified Newcombe Stratified Newcombe Newcombe and the Score(M-N) method, which is using the method described by Yan and Su[4]. See also Score(Miettinen-Nurminen) Score(M-N) 0.6 implemented in SAS 9.4 but was not available in r, along 0.85 0.85 0.85 0.85 Kim and Won[8]. 0.12 n_sig[, n_sig[, 1] with r code to run a simulation for implementing projected 0.4 Score (Miettinen-Nurminen) confidence limits sample sizes and event probabilities in 2 treatment arms. 0.2 0.80 0.80 0.80 • Stratum risk difference CIs are computed using the Entire range 0.80 Code available upon request. 0.08 Score method and combined using weights dependent 0.0 0.5 1.0 1.5 2.0 2.5 0.00 0.05 0.10 0.15 0.20 0.25 0.75 0.75 0.75 0.75 p_diff 0.95 p_diff[1:6] n_sig2[2, ] n_sig2[2, ] n_sig2[3, ] n_sig2[4, on the CI lengths. ] n_sig2[1, 0.60 References [1] Cochran, W.G. (1954). “Some methods for strengthening the common chi-square 0.70 0.70 0.70 0.70 tests”. Biometrics, 10, 417-451 0.85 Simulation 0.50 [2] Mantel, N., and Haenszel, W. (1959). “Statistical aspects of the analysis of data from Probability of significant 95% CI retrospective studies of disease”. Journal of the National Cancer Institute, 22, 719-748. 0.65 0.65 0.65 Data are simulated using 4 strata and an overall event 0.65 [3] Sato, T. (1989), “On the Variance Estimator of the Mantel-Haenszel Risk Difference,” probability similar to an actual trial. Across all simulations 0.75 Probability of significant 95% CI 0.40 Biometrics, 45, 1323–1324, letter to the editor. 0.60 0.60 0.60 the underlying overall event probability (poverall=2.5%) and 0.60 0.65 0.70 0.75 0.80 0.85 1.0 1.1 1.2 1.3 1.4 0.60 [4] Yan X. and Su X.G. (2010) “Stratified Wilson and Newcombe confidence intervals for (p -p )*100 multiple binomial proportions” Statistics in Medicine 19, 811-825 the sample sizes per stratum remain stable (n1j=2,350, p_diff[13:18] ct tx p_diff[20:30] *Heterogenity introduced by sampling from uniform distribution and adding to ptx 1:4 1:4 1:4 1:4 [5] Agresti, A. (2013), Categorical Data Analysis, 3rd Edition, Hoboken, NJ: John Wiley & n2j=125, n3j=1050, n4j=75, where j=1,2 for treatment or Sons. control group, i.e. using 1:1 allocation to treatment and Coverage probability over varying treatment differences p -p =-0.01 [6] Miettinen, O. S. and Nurminen, M. M. (1985), “Comparative Analysis of Two Rates,” Poverall=2.5% tx ct low heterogenity med heterogenity high heterogenity Statistics in Medicine, 4, 213–226 control). (no heterogenity) uni[-1e-04,1e-04] uni[-0.001,0.001] uni[-0.01,0.01] [7] Klingenberg, B. (2013) “A new and improved confidence interval for the Mantel- Wald(Sato) 1.00 1.00 1.00 1.00 New(Klingenberg) Haenszel risk difference” Statistics in Medicine, 33(17), 2968-83. MH Wald-type(Sato) Stratified Newcombe Sample sizes per strata per treatment Score(M-N) [8] Kim, Y., Won, S. (2013) “A new and improved confidence interval for the Mantel- 2350 New(Klingenberg) Haenszel risk difference,” in Proceedings of PharmaSUG 2013 (Pharmaceutical Industry Stratified Newcombe 0.98 Score(Miettinen-Nurminen) SAS Users Group), Cary, NC: SAS Institute Inc. [9] RStudio Team (2015). RStudio: Integrated Development for R. RStudio, Inc., Boston, 2000 MA URL http://www.rstudio.com/. 0.95 0.95 0.95 0.95 0.94 1500 Contact Information: 1050 n_est[, 1] n_est2[2, ] n_est2[2, ] n_est2[3, ] n_est2[4, n_est2[1, ] n_est2[1, [email protected] 1000 0.90 919-294-5562 0.90 0.90 0.90 0.90 PAREXEL International Corp 500 2520 Meridian Parkway, Suite 200 125 75 0.86 Durham, North Carolina 27713 0 0.0 0.5 1.0 1.5 2.0 2.5 Acknowledgements: Thank you to Daniel Bonzo, Benedict Dormitorio, and Young, Low risk Young, High risk Old, Low risk Old, High risk Probability true risk diff captured in 95% CI p =2.5% p =3.75% ct ct Probability true average risk diff is in 95% CI Marlina Nasution of PAREXEL for their contributions of ideas and edits. 0.85 0.85 0.85 p_diff 0.85 Randomization Strata ptx=2.5% (pct-ptx)*100 ptx=1.25% © 2015 PAREXEL International Corporation. All rights reserved. *Heterogenity introduced by sampling from uniform distribution and adding to ptx 1:4 1:4 1:4 1:4.
Load more

Recommended publications
  • Stratification Trees for Adaptive Randomization in Randomized
    Stratification Trees for Adaptive Randomization in Randomized Controlled Trials Max Tabord-Meehan Department of Economics Northwestern University [email protected] ⇤ (Click here for most recent version) 26th October 2018 Abstract This paper proposes an adaptive randomization procedure for two-stage randomized con- trolled trials. The method uses data from a first-wave experiment in order to determine how to stratify in a second wave of the experiment, where the objective is to minimize the variance of an estimator for the average treatment e↵ect (ATE). We consider selection from a class of stratified randomization procedures which we call stratification trees: these are procedures whose strata can be represented as decision trees, with di↵ering treatment assignment probabilities across strata. By using the first wave to estimate a stratification tree, we simultaneously select which covariates to use for stratification, how to stratify over these covariates, as well as the assign- ment probabilities within these strata. Our main result shows that using this randomization procedure with an appropriate estimator results in an asymptotic variance which minimizes the variance bound for estimating the ATE, over an optimal stratification of the covariate space. Moreover, by extending techniques developed in Bugni et al. (2018), the results we present are able to accommodate a large class of assignment mechanisms within strata, including stratified block randomization. We also present extensions of the procedure to the setting of multiple treatments, and to the targeting of subgroup-specific e↵ects. In a simulation study, we find that our method is most e↵ective when the response model exhibits some amount of “sparsity” with respect to the covariates, but can be e↵ective in other contexts as well, as long as the first-wave sample size used to estimate the stratification tree is not prohibitively small.
    [Show full text]
  • Pragmatic Cluster Randomized Trials Using Covariate Constrained Randomization: a Method for Practice-Based Research Networks (Pbrns)
    SPECIAL COMMUNICATION Pragmatic Cluster Randomized Trials Using Covariate Constrained Randomization: A Method for Practice-based Research Networks (PBRNs) L. Miriam Dickinson, PhD, Brenda Beaty, MSPH, Chet Fox, MD, Wilson Pace, MD, W. Perry Dickinson, MD, Caroline Emsermann, MS, and Allison Kempe, MD, MPH Background: Cluster randomized trials (CRTs) are useful in practice-based research network transla- tional research. However, simple or stratified randomization often yields study groups that differ on key baseline variables when the number of clusters is small. Unbalanced study arms constitute a potentially serious methodological problem for CRTs. Methods: Covariate constrained randomization with data on relevant variables before randomization was used to achieve balanced study arms in 2 pragmatic CRTs. In study 1, 16 counties in Colorado were randomized to practice-based or population-based reminder recall for vaccinating children ages 19 to 35 months. In study 2, 18 primary care practices were randomized to computer decision support plus practice facilitation versus computer decision support alone to improve care for patients with stage 3 and 4 chronic kidney disease. For each study, a set of optimal randomizations, which minimized differ- ences of key variables between study arms, was identified from the set of all possible randomizations. Results: Differences between study arms were smaller in the optimal versus remaining randomiza- tions. Even for the randomization in the optimal set with the largest difference between groups, study arms did not differ significantly on any variable for either study (P > .05). Conclusions: Covariate constrained randomization, which restricts the full randomization set to a subset in which differences between study arms are minimized, is a useful tool for achieving balanced study arms in CRTs.
    [Show full text]
  • Random Allocation in Controlled Clinical Trials: a Review
    J Pharm Pharm Sci (www.cspsCanada.org) 17(2) 248 - 253, 2014 Random Allocation in Controlled Clinical Trials: A Review Bolaji Emmanuel Egbewale Department of Community Medicine, Ladoke Akintola University of Technology, Ogbomoso, Nigeria Received, February 16, 2014; Revised, May 23, 2014; Accepted, May 30, 2014; Published, June 2, 2014. ABSTRACT- PURPOSE: An allocation strategy that allows for chance placement of participants to study groups is crucial to the experimental nature of randomised controlled trials. Following decades of the discovery of randomisation considerable erroneous opinion and misrepresentations of its concept both in principle and practice still exists. In some circles, opinions are also divided on the strength and weaknesses of each of the random allocation strategies. This review provides an update on various random allocation techniques so as to correct existing misconceptions on this all important procedure. METHODS: This is a review of literatures published in the Pubmed database on concepts of common allocation techniques used in controlled clinical trials. RESULTS: Allocation methods that use; case record number, date of birth, date of presentation, haphazard or alternating assignment are non-random allocation techniques and should not be confused as random methods. Four main random allocation techniques were identified. Minimisation procedure though not fully a random technique, however, proffers solution to the limitations of stratification at balancing for multiple prognostic factors, as the procedure makes treatment groups similar in several important features even in small sample trials. CONCLUSIONS: Even though generation of allocation sequence by simple randomisation procedure is easily facilitated, a major drawback of the technique is that treatment groups can by chance end up being dissimilar both in size and composition of prognostic factors.
    [Show full text]
  • In Pursuit of Balance
    WPS4752 POLICY RESEA R CH WO R KING PA P E R 4752 Public Disclosure Authorized In Pursuit of Balance Randomization in Practice Public Disclosure Authorized in Development Field Experiments Miriam Bruhn David McKenzie Public Disclosure Authorized The World Bank Public Disclosure Authorized Development Research Group Finance and Private Sector Team October 2008 POLICY RESEA R CH WO R KING PA P E R 4752 Abstract Randomized experiments are increasingly used in emerge. First, many researchers are not controlling for the development economics, with researchers now facing method of randomization in their analysis. The authors the question of not just whether to randomize, but show this leads to tests with incorrect size, and can result how to do so. Pure random assignment guarantees that in lower power than if a pure random draw was used. the treatment and control groups will have identical Second, they find that in samples of 300 or more, the characteristics on average, but in any particular random different randomization methods perform similarly in allocation, the two groups will differ along some terms of achieving balance on many future outcomes of dimensions. Methods used to pursue greater balance interest. However, for very persistent outcome variables include stratification, pair-wise matching, and re- and in smaller sample sizes, pair-wise matching and randomization. This paper presents new evidence on stratification perform best. Third, the analysis suggests the randomization methods used in existing randomized that on balance the re-randomization methods common experiments, and carries out simulations in order to in practice are less desirable than other methods, such as provide guidance for researchers.
    [Show full text]
  • Week 10: Causality with Measured Confounding
    Week 10: Causality with Measured Confounding Brandon Stewart1 Princeton November 28 and 30, 2016 1These slides are heavily influenced by Matt Blackwell, Jens Hainmueller, Erin Hartman, Kosuke Imai and Gary King. Stewart (Princeton) Week 10: Measured Confounding November 28 and 30, 2016 1 / 176 Where We've Been and Where We're Going... Last Week I regression diagnostics This Week I Monday: F experimental Ideal F identification with measured confounding I Wednesday: F regression estimation Next Week I identification with unmeasured confounding I instrumental variables Long Run I causality with measured confounding ! unmeasured confounding ! repeated data Questions? Stewart (Princeton) Week 10: Measured Confounding November 28 and 30, 2016 2 / 176 1 The Experimental Ideal 2 Assumption of No Unmeasured Confounding 3 Fun With Censorship 4 Regression Estimators 5 Agnostic Regression 6 Regression and Causality 7 Regression Under Heterogeneous Effects 8 Fun with Visualization, Replication and the NYT 9 Appendix Subclassification Identification under Random Assignment Estimation Under Random Assignment Blocking Stewart (Princeton) Week 10: Measured Confounding November 28 and 30, 2016 3 / 176 Lancet 2001: negative correlation between coronary heart disease mortality and level of vitamin C in bloodstream (controlling for age, gender, blood pressure, diabetes, and smoking) Stewart (Princeton) Week 10: Measured Confounding November 28 and 30, 2016 4 / 176 Lancet 2002: no effect of vitamin C on mortality in controlled placebo trial (controlling for nothing) Stewart (Princeton) Week 10: Measured Confounding November 28 and 30, 2016 4 / 176 Lancet 2003: comparing among individuals with the same age, gender, blood pressure, diabetes, and smoking, those with higher vitamin C levels have lower levels of obesity, lower levels of alcohol consumption, are less likely to grow up in working class, etc.
    [Show full text]
  • How to Do Random Allocation (Randomization) Jeehyoung Kim, MD, Wonshik Shin, MD
    Special Report Clinics in Orthopedic Surgery 2014;6:103-109 • http://dx.doi.org/10.4055/cios.2014.6.1.103 How to Do Random Allocation (Randomization) Jeehyoung Kim, MD, Wonshik Shin, MD Department of Orthopedic Surgery, Seoul Sacred Heart General Hospital, Seoul, Korea Purpose: To explain the concept and procedure of random allocation as used in a randomized controlled study. Methods: We explain the general concept of random allocation and demonstrate how to perform the procedure easily and how to report it in a paper. Keywords: Random allocation, Simple randomization, Block randomization, Stratified randomization Randomized controlled trials (RCT) are known as the best On the other hand, many researchers are still un- method to prove causality in spite of various limitations. familiar with how to do randomization, and it has been Random allocation is a technique that chooses individuals shown that there are problems in many studies with the for treatment groups and control groups entirely by chance accurate performance of the randomization and that some with no regard to the will of researchers or patients’ con- studies are reporting incorrect results. So, we will intro- dition and preference. This allows researchers to control duce the recommended way of using statistical methods all known and unknown factors that may affect results in for a randomized controlled study and show how to report treatment groups and control groups. the results properly. Allocation concealment is a technique used to pre- vent selection bias by concealing the allocation sequence CATEGORIES OF RANDOMIZATION from those assigning participants to intervention groups, until the moment of assignment.
    [Show full text]
  • Guideline on Adjustment for Baseline Covariates' (EMA/CHMP/295050/2013)
    26 February 2015 EMA/40143/2014 CHMP Biostatistics Working Party (BSWP) Overview of comments received on 'Guideline on adjustment for baseline covariates' (EMA/CHMP/295050/2013) Interested parties (organisations or individuals) that commented on the draft document as released for consultation. Stakeholder no. Name of organisation or individual 1 Alisa J. Stephens, University of Pennsylvania, USA; Eric J. Tchetgen Tchetgen, Harvard University, USA; Victor De Gruttola, Harvard University, USA. 2 European Federation of Statisticians in the Pharmaceutical Industry (EFSPI) 3 Merck Sharp & Dohme (MSD) 4 F. Hoffmann – la Roche Ltd. 5 The European Federation of Pharmaceutical Industries and Associations (EFPIA) 6 Teva Pharmaceutical Ltd 7 Ablynx (Heidi Wouters en Katrien Verschueren) 8 IDeAl (Integrated Design and AnaLysis of small population group trials) FP7 Consortium 9 SciencePharma (Poland) 30 Churchill Place ● Canary Wharf ● London E14 5EU ● United Kingdom Telephone +44 (0)20 3660 6000 Facsimile +44 (0)20 3660 5555 Send a question via our website www.ema.europa.eu/contact An agency of the European Union © European Medicines Agency, 2015. Reproduction is authorised provided the source is acknowledged. 1. General comments – overview Stakeholder no. General comment (if any) Outcome (if applicable) 1 1. Introduction The proposed EMA guidelines highlight important issues surrounding covariate adjustment in the analysis and interpretation of results from randomized trials. As the guidelines state, proper covariate adjustment can enhance precision in the estimation of treatment effects; however, doing so in practice raises several important issues, including interpretation of the estimated effects, evaluation of the impact of model misspecification, and the consequences of inclusion of postrandomization covariates.
    [Show full text]
  • Regression Analysis for Covariate-Adaptive Randomization: a Robust and Efficient Inference Perspective
    Regression analysis for covariate-adaptive randomization: A robust and efficient inference perspective Wei Ma1, Fuyi Tu1, Hanzhong Liu2∗ 1 Institute of Statistics and Big Data, Renmin University of China, Beijing, China 2 Center for Statistical Science, Department of Industrial Engineering, Tsinghua University, Beijing, China Abstract Linear regression is arguably the most fundamental statistical model; however, the validity of its use in randomized clinical trials, despite being common practice, has never been crystal clear, particularly when stratified or covariate-adaptive randomization is used. In this paper, we investigate several of the most intuitive and commonly used regression models for estimating and inferring the treatment effect in randomized clinical trials. By allowing the regression model to be arbitrarily misspecified, we demonstrate that all these regression-based estimators robustly estimate the treatment effect, albeit with possibly different efficiency. We also propose consistent non-parametric variance estimators and compare their performances to those of the model-based variance estimators that are readily available in standard statistical software. Based on the results and taking into account both theoretical efficiency and practical feasibility, we make recommendations for the effective use of regression under various scenarios. For equal allocation, it suffices to use the regression adjustment for the stratum covariates and additional baseline covariates, if available, with the usual ordinary-least-squares variance estimator. For arXiv:2009.02287v1 [stat.ME] 4 Sep 2020 unequal allocation, regression with treatment-by-covariate interactions should be used, together with our proposed variance estimators. These recommendations apply to simple and stratified randomization, and minimization, among others. We hope this work helps to clarify and promote the usage of regression in randomized clinical trials.
    [Show full text]
  • Adjusted Proportion Difference and Confidence Interval In
    PharmaSUG 2013 - Paper SP04 Adjusted proportion difference and confidence interval in stratified randomized trials Yeonhee Kim, Gilead Sciences, Seattle, WA Seunghyun Won, University of Pittsburgh, Pittsburgh, PA ABSTRACT Stratified randomization is widely used in clinical trials to achieve treatment balance across strata. In the analysis, investigators are often interested in the estimation of common treatment effect adjusting for stratification factors. We developed the macro for stratified analysis when the primary endpoint is a difference in proportions between two treatment groups. The %strataCI macro estimates proportion difference using three different weighting schemes and calculates Wald and Newcombe confidence intervals. Keywords: Stratified analysis, weight-adjusted proportion, Minimum risk weight, Wald, Newcombe confidence interval INTRODUCTION Stratified randomization is often used when treatment outcome is presumed to be affected by baseline characteristics such as gender and disease severity. Another example of stratification can be found in multicenter trials in which patients are randomly assigned to the treatments within each center. In the analysis, treatment effects are not only assessed separately by each center but also combined after adjusting for center effect. This paper focuses on the stratified analysis for binary outcomes (success or failure) when the primary endpoint is a proportion difference between two treatment groups. Starting from the stratum-specific proportion difference, we consider the point estimate and corresponding confidence interval for the strata-adjusted proportion difference. The adjusted effect is obtained by weighted average of stratum specific rate differences. For treatment comparison, we present Wald type confidence interval as well as stratified Newcombe confidence interval. METHOD STRATUM-SPECIFIC PROPORTION DIFFERENCE Let pij is the proportion of success in treatment i (i = 1, 2) and stratum j (j = 1, ..., s), then the stratum-specific th proportion difference is defined as dj = p1j – p2j.
    [Show full text]
  • Randomization Lists
    PASS Sample Size Software NCSS.com Chapter 880 Randomization Lists Introduction This module is used to create a randomization list for assigning subjects to one of up to 25 treatment groups. The list can be stratified by up to 25 centers (strata) with an additional 2 stratification factors, each with up to 25 levels. Seven randomization algorithms are available. Five of the algorithms (Block Randomization, Efron’s biased coin randomization, Smith’s randomization, Wei’s urn randomization, and random sorting using maximum allowable % deviation) are designed to generate balanced random samples throughout the course of an experiment. The other two (complete randomization and random sorting) are less complex but have higher probabilities of imbalance. Technical details for each of these algorithms (except random sorting) as well as a discussion of randomization issues in general are given in Rosenberger and Lachin (2002), Pocock (1983), and Piantadosi (2005). Why Randomize? Random allocation has been an important part of scientific discovery since the time of R.A. Fisher. He and other scientists recognized the potential for bias in nonrandomized experiments that could compromise treatment comparisons. Randomization controls the influence of unknown factors, reducing potential bias in an experiment. Rosenberger and Lachin (2002) state on page 18 that “…the randomization of subjects between the treatment groups is the paramount statistical element that allows one to claim that a study is unbiased.” The elimination of bias, however, is not the only reason for randomization. Pocock (1983) states on page 50 that randomization is used “to provide a basis for the standard methods of statistical analysis such as significance tests.” Clearly, randomization is necessary in order to produce accurate and generalizable statistical conclusions.
    [Show full text]
  • [Stat.ME] 3 Jul 2016 the Econometrics of Randomized Experiments
    The Econometrics of Randomized Experiments∗ Susan Athey† Guido W. Imbens‡ Current version July 2016 Abstract In this chapter, we present econometric and statistical methods for analyzing ran- domized experiments. For basic experiments we stress randomization-based inference as opposed to sampling-based inference. In randomization-based inference, uncertainty in es- timates arises naturally from the random assignment of the treatments, rather than from hypothesized sampling from a large population. We show how this perspective relates to regression analyses for randomized experiments. We discuss the analyses of stratified, paired, and clustered randomized experiments, and we stress the general efficiency gains from stratification. We also discuss complications in randomized experiments such as non- compliance. In the presence of non-compliance we contrast intention-to-treat analyses with instrumental variables analyses allowing for general treatment effect heterogeneity. We con- sider in detail estimation and inference for heterogenous treatment effects in settings with (possibly many) covariates. These methods allow researchers to explore heterogeneity by identifying subpopulations with different treatment effects while maintaining the ability to construct valid confidence intervals. We also discuss optimal assignment to treatment based on covariates in such settings. Finally, we discuss estimation and inference in exper- iments in settings with interactions between units, both in general network settings and in settings where the population is partitioned into groups with all interactions contained within these groups. arXiv:1607.00698v1 [stat.ME] 3 Jul 2016 JEL Classification: C01, C13, C18, C21, C52, C54 Keywords: Regression Analyses, Random Assignment, Randomized Experiments, Potential Outcomes, Causality ∗We are grateful for comments by Esther Duflo. †Professor of Economics, Graduate School of Business, Stanford University, and NBER, [email protected].
    [Show full text]
  • EVALUATING ALTERNATIVE STRATEGIES for STRATIFICATION FACTORS in RANDOMIZED CONTROLLED TRIAL ANALYSIS By
    EVALUATING ALTERNATIVE STRATEGIES FOR STRATIFICATION FACTORS IN RANDOMIZED CONTROLLED TRIAL ANALYSIS by Stuart Alan Gansky Department of Biostatistics University of North Carolina Institute of Statistics Mimeo Series No. 2168T August 1996 EVALUATING ALTERNATIVE STRATEGIES FOR STRATIFICATION FACTORS IN RANDOMIZED CONTROLLED TRIAL ANALYSIS by Stuart Alan Gansky A Paper submitted to the faculty of the University ofNorth Carolina at Chapel Hill in partial fulfillment ofthe requirements for the degree ofDoctor ofPublic Health in the Department ofBiostatistics Chapel Hill 1996 Approved by: Advisor Reader ----:::::r~c:....::....::.=.o....-=+---.::..::..-=s._ +-=~~::::::::::::~~~ Reader 111 EVALUATING ALTERNATIVE STRATEGIES FOR STRATIFICATION FACTORS IN RANDOMIZED CONTROLLED TRIAL ANALYSIS Stuart Alan Gansky (under the direction ofGary Grove Koch) ABSTRACT In randomized controlled trials, traditionally there are a set of strata and within each stratum patients are randomized to a treatment condition. Stratified designs are often used to address concerns about study generalizability to traditionally underrepresented patient subgroups. For many studies, the stratification applied during design is much more extensive than can be fully incorporated in analyses. One potential strategy for such situations is to ignore one or more stratification factors; another is to account for one or more factors with nonparametric covariance adjustment. Consequences ofignoring strata or adjusting strata as covariates, relative to fully stratified procedures are not completely known and deserve further investigation, since trialists frequently perform analyses not fully accounting for stratification. This research evaluates properties of strategies that ignore strata or adjust strata as covariates using both design- and model-based methods, relative to fully stratified analyses, in tenns ofvalidity and power. Derivations, simulations and actual examples illustrate the perfonnance in various scenarios.
    [Show full text]