<<

SISCR UW - 2016

Introduction to Clinical Trials - Day 2 Why randomization? Session 3 - Methods of Randomization Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted Presented July 26, 2016 estimators Nonadaptive Randomization Complete randomization Blocked randomization Stratified randomization

Adaptive Randomization Covariate adaptive Susanne J. May randomization Response adaptive Department of randomization University of Washington Logistics of Randomization Daniel L. Gillen Department of University of California, Irvine

c 2016 Daniel L. Gillen, PhD and Susanne J. May, PhD SISCR - RCT, Day 2 - 3 : 1

Why randomization? SISCR UW - 2016

Why randomization? Consider the scientific objective Bias Motivating example: Smoking & FEV I ICH guidelines (www.ich.org) part E9 Statistical Principles Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted “The most important design techniques for avoiding bias in estimators clinical trials are blinding and randomisation, and these Nonadaptive Randomization should be normal features of most controlled clinical trials Complete randomization Blocked randomization intended to be included in a marketing application." Stratified randomization Adaptive Randomization I Covariate adaptive Similar criteria are required in the CONSORT guidelines. randomization Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 : 2 Bias SISCR UW - 2016 What is bias?

I Bias is a tendency of a statistical estimate to deviate in Why randomization? one direction from a“true value" Bias Motivating example: Smoking & FEV Statistical role of variables I What defines the “truth" is dictated by the scientific goal Adjusted vs. unadjusted effects Precision of adjusted estimators

I Randomization is the primary tool of a clinical trialist for Nonadaptive Randomization reducing bias Complete randomization Blocked randomization Stratified randomization

I In order to illustrate the role in which bias arises in clinical Adaptive Randomization studies and motivate the role of randomization, it is useful Covariate adaptive to review the components of a in order to: randomization Response adaptive randomization

Logistics of 1. Develop a standard nomenclature Randomization 2. Illustrate the goals and impact of randomization

I To this end, we can begin withe role of adjustment variables in statistical models

SISCR - RCT, Day 2 - 3 : 3

Example - FEV Data SISCR Is there an association between smoking and lung function in UW - 2016 children?

I Scientific justification Why randomization? Bias Motivating example: Smoking & FEV I Longterm smoking is associated with lower lung function Statistical role of variables Adjusted vs. unadjusted effects I Precision of adjusted Are similar effects observed in short term smoking in estimators

children? Nonadaptive Randomization Complete randomization Blocked randomization Stratified randomization I Causal pathway of interest Adaptive Randomization Covariate adaptive I Interested in whether smoking will cause a decrease in lung randomization Response adaptive function randomization Logistics of Randomization

Smoking- Lung function

SISCR - RCT, Day 2 - 3 : 4 Example - FEV Data SISCR UW - 2016

Study design Why randomization? Bias I Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Measurements obtained on a sample of 654 healthy effects Precision of adjusted children estimators Nonadaptive Randomization I Children were sampled while being seen for a regular Complete randomization checkup Blocked randomization Stratified randomization

Adaptive I Data available on smoking, age, gender, and height Randomization Covariate adaptive randomization Response adaptive I Predictor of interest: Self-reported smoking randomization

Logistics of Randomization I Response: FEV (Forced Expository Volume)

SISCR - RCT, Day 2 - 3 : 5

FEV Data SISCR

SMOKERS UW - 2016

1.953 2.236 3.428 3.208 1.694 3.957 4.789 2.384 3.074 2.387 3.835 2.599 4.756 3.086 4.309 3.413 2.975 3.169 3.343 3.751 2.216 3 .078 3.186 3.297 2.304 3.102 2.677 3.297 3.498 2.759 2.953 3.785 2.276 4.637 3.038 3.120 3.339 3.152 3.104 4.045 4.763 3.069 4.506 3.519 3.688 2.679 2.198 3 .345 3.082 2.903 3.004 3.406 3.122 3.330 2.608 3.799 4.086 4.070 2.264 4.404 Why randomization? 2.278 4.872 4.270 3.727 2.795 Bias Motivating example: Smoking & FEV NONSMOKERS Statistical role of variables Adjusted vs. unadjusted effects 1.708 1.724 1.720 1.558 1.895 2.336 1.919 1.415 1.987 1.942 1.602 1.735 2.193 2.118 2.258 1.932 1.472 1.878 2.352 2.604 1.400 1 .256 0.839 2.578 2.988 1.404 2.348 1.755 2.980 2.100 1.282 3.000 2.673 2.093 1.612 2.175 2.725 2.071 1.547 2.004 Precision of adjusted estimators 3.135 2.420 1.776 1.931 1.343 2.076 1.624 1 .344 1.650 2.732 2.017 2.797 3.556 1.703 1.634 2.570 3.016 2.419 1.569 1.698 2.123 2.481 1.481 1.577 1.940 1.747 2.069 1.631 1.536 2.560 1.962 2.531 2.715 2 .457 2.090 1.789 1.858 1.452 3.842 1.719 Nonadaptive 2.111 1.695 2.211 1.794 1.917 2.144 1.253 2.659 1.580 2.126 3.029 2.964 1.611 2.215 2.388 2.196 1.751 2.165 1.682 1 .523 Randomization 1.292 1.649 2.588 0.796 2.574 1.979 2.354 1.718 1.742 1.603 2.639 1.829 2.084 2.220 1.473 2.341 1.698 1.196 1.872 2.219 Complete randomization 2.420 1.827 1.461 1.338 2.090 1 .697 1.562 2.040 1.609 2.458 2.650 1.429 1.675 1.947 2.069 1.572 1.348 2.288 1.773 0.791 1.905 2.463 1.431 2.631 3.114 2.135 1.527 2.293 3.042 2.927 2.665 2 .301 2.460 2.592 1.750 1.759 1.536 2.259 2.048 2.571 Blocked randomization 2.046 1.780 1.552 1.953 2.893 1.713 2.851 1.624 2.631 1.819 1.658 2.158 1.789 3.004 2.503 1.933 2.091 2 .316 1.704 1.606 Stratified randomization 1.165 2.102 2.320 2.230 1.716 1.790 1.146 2.187 2.717 1.796 1.335 2.119 1.666 1.826 2.709 2.871 1.092 2.262 2.104 2.166 Adaptive 1.690 2.973 2.145 1 .971 2.095 1.697 2.455 1.920 2.164 2.130 2.993 2.529 1.726 2.442 1.102 2.056 1.808 2.305 1.969 1.556 Randomization 1.072 2.042 1.512 1.423 3.681 1.991 1.897 1.370 1.338 2 .016 2.639 1.389 1.612 2.135 2.681 3.223 1.796 2.010 1.523 1.744 2.485 2.335 1.415 2.076 2.435 1.728 2.850 1.844 1.754 1.343 2.303 2.246 2.476 3.239 2.457 2 .382 1.640 1.589 2.056 2.226 Covariate adaptive randomization 1.886 2.833 1.715 2.631 2.550 1.912 1.877 1.935 1.539 2.803 2.923 2.358 2.094 1.855 1.535 2.135 1.930 2.182 1.359 2.002 1.699 2 .500 2.366 2.069 1.418 2.333 1.514 1.758 2.535 2.564 2.487 1.591 1.624 2.798 1.691 1.999 1.869 1.004 1.427 1.826 Response adaptive randomization 2.688 1.657 1.672 2.015 2.371 2.115 2.328 1 .495 2.884 2.328 3.381 2.170 3.470 3.058 1.811 2.524 2.642 3.741 4.336 4.842 4.550 2.841 3.166 3.816 2.561 3.654 2.481 2.665 3.203 3.549 3.222 3.111 3.490 3 .147 2.520 2.292 2.889 2.246 1.937 2.646 Logistics of 2.957 4.007 2.386 3.251 2.762 3.011 4.305 3.906 3.583 3.236 3.436 3.058 3.007 3.489 2.864 2.819 2.250 4.683 2.352 3 .108 Randomization 3.994 4.393 2.592 3.193 2.346 3.515 2.754 2.720 2.463 2.633 3.048 3.111 3.745 2.094 3.183 3.977 3.354 3.411 3.171 3.887 2.646 2.504 3.587 3.845 2.971 2 .891 1.823 2.417 2.175 2.735 4.273 2.976 4.065 2.318 3.596 3.395 2.751 2.673 2.556 2.542 2.608 2.354 1.458 3.795 2.491 3.060 2.545 2.993 3.305 3.774 2.855 2 .988 2.498 3.169 2.887 2.704 3.515 3.425 2.287 2.434 2.365 2.696 2.868 2.813 3.255 4.593 4.111 1.916 1.858 3.350 2.901 2.241 4.225 3.223 5.224 4.073 4.080 2 .606 4.411 3.791 3.089 2.465 3.200 2.913 4.877 2.358 3.279 2.581 2.347 2.691 2.827 1.873 2.538 2.758 3.050 3.079 2.201 1.858 3.403 3.501 2.578 1.665 2.081 2 .974 4.073 4.448 3.984 2.250 2.752 3.680 2.862 3.023 3.681 3.255 3.692 2.356 4.591 3.082 3.258 2.216 3.247 4.324 2.362 2.563 3.206 3.585 4.720 3.331 5.083 2 .417 2.364 2.341 3.231 3.078 3.369 3.529 2.866 2.891 3.022 3.127 2.866 2.605 3.056 2.569 2.501 3.320 2.123 3.780 3.847 3.924 2.132 2.752 2.449 3.456 3.073 2 .688 3.329 4.271 3.530 2.928 2.689 2.332 2.934 3.110 2.894 2.435 2.838 3.035 4.831 2.812 2.714 3.086 3.519 4.232 2.770 3.341 3.090 2.531 2.822 2.935 2.568 2 .387 2.499 4.130 3.001 3.132 3.577 3.222 3.280 2.659 2.822 2.140 4.203 2.997 2.562 3.082 3.806 2.458 2.391 3.141 2.579 2.100 2.785 4.284 2.906 5.102 4.429 4 .279 4.500 2.635 3.082 3.387 5.793 3.985 4.220 4.724 3.731 3.500 3.674 5.633 SISCR - RCT, Day 2 - 3 : 6 3.645 2.887 3.960 4.299 2.981 4.504 5.638 2.853 3.211 Example - FEV Data SISCR UW - 2016

Why randomization? Bias Interpretation of smoking effect in unadjusted analysis Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Restrict sample to children 9 years and above (age of effects Precision of adjusted youngest smoker in sample) estimators Nonadaptive Randomization I Consider log-transformation of FEV based upon past Complete randomization Blocked randomization studies Stratified randomization

I Scientific focus on FEV Adaptive Randomization I Distribution of log-transformed FEV approximately Covariate adaptive randomization symmetric Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 : 7

Example - FEV Data SISCR UW - 2016

Why randomization? Bias Motivating example: Smoking & FEV Unadjusted association between smoking and FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I Consider an unadjusted comparison of FEV between estimators

smokers and non-smokers Nonadaptive Randomization Complete randomization I Unadjusted Result: The median FEV of a smoker is Blocked randomization estimated to be 10.8% higher than that of a non-smoker Stratified randomization Adaptive (95% CI: 1.04, 1.18). This difference is statistically Randomization Covariate adaptive significant p = 0.002. randomization Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 : 8 Example - FEV Data SISCR UW - 2016

Why randomization? Bias Motivating example: Smoking & FEV Adjustment for age Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I Consider adjustment for age in a model estimators

Nonadaptive Randomization I Age-adjusted result: The median FEV of a smokers is Complete randomization estimated to be 5.0% lower than that of non-smokers similar Blocked randomization in age (95% CI: 0.90, 1.01). This difference is not Stratified randomization Adaptive statistically significant at the .05 level (p = 0.093). Randomization Covariate adaptive randomization Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 : 9

Example - FEV Data SISCR UW - 2016

Adjustment for age and height Why randomization? Bias Motivating example: Smoking & FEV I After adjustment for age, height should have little Statistical role of variables Adjusted vs. unadjusted association with smoking status but is still likely to have an effects Precision of adjusted association with FEV. estimators

Nonadaptive Randomization I Consider additional adjustment for height... Complete randomization Blocked randomization Stratified randomization

I Age and height-adjusted result: The median FEV of Adaptive smokers is estimated to be 5.2% lower than that of Randomization Covariate adaptive non-smokers similar in age and height (95% CI: 0.91, 0.99). randomization Response adaptive This difference is statistically significant at the .05 level randomization (p = 0.011). Logistics of Randomization

SISCR - RCT, Day 2 - 3 :10 Example - FEV Data SISCR UW - 2016

Comparison of age and age-height adjusted analyses Why randomization? Bias Motivating example: Smoking & FEV I Notice that there is little difference in estimated effect of Statistical role of variables Adjusted vs. unadjusted smoking between age adjusted models with and without effects Precision of adjusted height estimators

Nonadaptive Randomization I Effect of height adjustment on precision Complete randomization Blocked randomization Stratified randomization

I Lower Root MSE (.144 vs .209) in height adjusted model Adaptive resulting in increased precision of estimate of smoking Randomization Covariate adaptive effect randomization Response adaptive randomization

I Net effect: Much greater precision (SE 0.021 vs 0.031) Logistics of Randomization

SISCR - RCT, Day 2 - 3 :11

Example - FEV Data SISCR UW - 2016

Why randomization? Take-home message Bias Motivating example: Smoking & FEV Statistical role of variables I Our scientific question was not Adjusted vs. unadjusted effects Precision of adjusted estimators “Is there a difference between smokers’ and nonsmokers’ Nonadaptive median FEV?" Randomization Complete randomization Blocked randomization Stratified randomization I But rather Adaptive Randomization Covariate adaptive “Do smokers have lower median FEV than otherwise randomization Response adaptive comparable nonsmokers?" randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 :12 Example - FEV Data SISCR UW - 2016 Take-home message

I This example highlights: Why randomization? Bias Motivating example: 1. How a scientific question should dictate a chosen statistical Smoking & FEV Statistical role of variables model Adjusted vs. unadjusted effects Precision of adjusted 2. The role of a variable on association estimates estimators Nonadaptive Randomization 3. The impact that adjustment has on the precision of Complete randomization Blocked randomization association estimates Stratified randomization Adaptive Randomization Covariate adaptive I These ideas provide the motivation for randomization, as randomization Response adaptive well as the types and implementation of various randomization

randomization methods Logistics of Randomization

I However, before going there, it is useful to define the statsitical role of variables and to generalize the observations that were made in the FEV example...

SISCR - RCT, Day 2 - 3 :13

Statistical role of variables SISCR UW - 2016

Effect modifiers ( terms) Why randomization? Bias Motivating example: I Suppose that we are interested in modeling the Smoking & FEV Statistical role of variables association between an outcome variable Y and a Adjusted vs. unadjusted effects predictor X Precision of adjusted estimators

Nonadaptive I Consider four broad categories of variables (this Randomization Complete randomization terminology is not universal) Blocked randomization Stratified randomization

Adaptive Randomization Covariate adaptive I Effect modifiers (interaction variables) randomization Response adaptive randomization

I An effect modifier (W ) is a covariate for which the Logistics of association between the predictor of interest (X) and the Randomization outcome of interest (Y ) differs with each level of W

SISCR - RCT, Day 2 - 3 :14 Statistical role of variables SISCR UW - 2016 Example: Effect modification

Why randomization? I Example: The association between gender and the risk of Bias Motivating example: chd differs by systolic blood pressure Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators ------sbpgrp | Odds Ratio chi2(1) P>chi2 [95% Conf. Interval] Nonadaptive ------+------Randomization 1 | 0.394493 86.23 0.0000 0.32186 0.48351 Complete randomization 2 | 0.429583 56.59 0.0000 0.34243 0.53892 Blocked randomization 3 | 0.597384 9.91 0.0016 0.43193 0.82621 Stratified randomization 4 | 0.741269 1.75 0.1858 0.47495 1.15693 ------Adaptive Randomization Covariate adaptive randomization Response adaptive randomization How do we deal with effect modifiers? Logistics of Randomization

I When the scientific question involves effect modification, analyses must be within each stratum separately

SISCR - RCT, Day 2 - 3 :15

Statistical role of variables SISCR Confounders UW - 2016

I One definition:Aconfounder is a variable that is causally Why randomization? related to the predictor of interest (X) and the outcome of Bias Motivating example: interest (Y ). Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators

Nonadaptive Randomization Complete randomization - Blocked randomization Predictor (X) Outcome (Y ) Stratified randomization

Adaptive HYH * Randomization H H Covariate adaptive H randomization H Response adaptive randomization

Logistics of Confounder (W ) Randomization

SISCR - RCT, Day 2 - 3 :16 Statistical role of variables SISCR UW - 2016

Example: Confounding Why randomization? Bias Motivating example: I Example: Age in the FEV example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I Older kids tend to smoke Precision of adjusted estimators

Nonadaptive I Older kids tend to have larger lungs Randomization Complete randomization Blocked randomization Stratified randomization

Adaptive Randomization Covariate adaptive randomization How do we deal with confounding? Response adaptive randomization

Logistics of I Adjust for the confounder Randomization

SISCR - RCT, Day 2 - 3 :17

Statistical role of variables SISCR UW - 2016 Precision variables

I I define a precision variable as a covariate that is related Why randomization? Bias to the outcome Y , but independent of the predictor of Motivating example: Smoking & FEV interest X. Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators

Nonadaptive Randomization Predictor (X)- Outcome (Y ) Complete randomization Blocked randomization Stratified randomization * Adaptive Randomization Covariate adaptive randomization Response adaptive Precision Variable (W ) randomization Logistics of Randomization

SISCR - RCT, Day 2 - 3 :18 Statistical role of variables SISCR Example: Precision variable UW - 2016

I Example: Height (after adjustment for age) in the FEV Why randomization? example: Bias Motivating example: Smoking & FEV Statistical role of variables I Conditional on age, little difference in prevalence of smoking Adjusted vs. unadjusted effects by height Precision of adjusted estimators

Nonadaptive I Conditional on age, taller kids tend to have larger lungs Randomization Complete randomization Blocked randomization Stratified randomization

Adaptive Randomization Covariate adaptive How do we deal with precision variables? randomization Response adaptive randomization I Often a good idea to control for them Logistics of Randomization I For example, in a two sample comparison of , we might control some variable in order to decrease the within group variability

SISCR - RCT, Day 2 - 3 :19

Statistical role of variables SISCR “Upstream" variables UW - 2016

I I define an upstream variable as a covariate that is Why randomization? independent of the outcome Y , but may or may not be Bias Motivating example: related to the predictor of interest X. Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators

Nonadaptive Predictor (X)- Outcome (Y ) Randomization Complete randomization Blocked randomization HY Stratified randomization HH H Adaptive H Randomization H Covariate adaptive randomization Response adaptive Upstream Variable (W ) randomization

Logistics of Randomization

I Generally a bad idea to adjust for “upstream" variables SISCR - RCT, Day 2 - 3 :20 Statistical role of variables SISCR UW - 2016 Why randomize?

Why randomization? I The fundamental statistical distinctions between Bias Motivating example: unadjusted and adjusted regression models are central to Smoking & FEV the goals of randomization Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I We thus want to be able to consider the relationships estimators Nonadaptive between Randomization Complete randomization Blocked randomization I unadjusted and adjusted parameters, and Stratified randomization I the standard errors of the two parameter estimates Adaptive Randomization Covariate adaptive randomization I This is easily done in the context of linear regression and Response adaptive randomization

that will be the setting for our discussion Logistics of Randomization

I Results are less straightforward for non-linear models (eg. logistic regression or proportional hazards) I However, the general principles still apply

SISCR - RCT, Day 2 - 3 :21

Statistical role of variables SISCR UW - 2016

Why randomization? Adjusted vs. unadjusted covariate effects Bias Motivating example: Smoking & FEV I Consider the following linear regression models: Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted 1. Unadjusted model: E[Yi ]=0 + 1Xi estimators

Nonadaptive Randomization I 1 is the difference in the of Y for groups differing by Complete randomization 1-unit in X Blocked randomization Stratified randomization

Adaptive 2. Adjusted model: E[Yi ]=0 + 1Xi + 2Wi Randomization Covariate adaptive randomization I Response adaptive 1 is the difference in the mean of Y for groups differing by randomization 1-unit in X, but agreeing in their value of W Logistics of Randomization

SISCR - RCT, Day 2 - 3 :22 Statistical role of variables SISCR UW - 2016 Adjusted vs. unadjusted covariate effects ˆ Why randomization? I Proposition 1: Let 1 denote the OLS estimate of 1. Then Bias Motivating example: under the adjusted model, Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted cov(X, W ) estimators [ˆ ]= + E 1 1 2 Nonadaptive var(X) Randomization Complete randomization Blocked randomization Stratified randomization var(W ) Adaptive = 1 + rXW 2 Randomization var(X) Covariate adaptive s randomization Response adaptive randomization

Logistics of Randomization where rXW , var(X), and var(W ) are the sample correlation between X and W , sample of X, and sample variance of W , respectively.

SISCR - RCT, Day 2 - 3 :23

Statistical role of variables SISCR The implication... UW - 2016 ˆ I 1 is biased (and inconsistent) for 1 unless at least one of the following hold Why randomization? Bias Motivating example: Smoking & FEV 1. rXW = 0:X and W are uncorrelated (in the sample), OR Statistical role of variables Adjusted vs. unadjusted 2. 2 = 0 : W is not related to Y effects Precision of adjusted estimators ˆ Nonadaptive I In either case, 1 is unbiased (and consistent) for 1 Randomization Complete randomization Blocked randomization I Implication for confounders? Stratified randomization Adaptive Randomization I By definition, a confounder is related to the predictor of Covariate adaptive randomization interest and the response Response adaptive randomization

Logistics of I This implies that if W is a confounder, then both conditions Randomization above fail

I Hence the parameter from the reduced model is biased for the adjusted estimate

SISCR - RCT, Day 2 - 3 :24 Precision of Estimators SISCR UW - 2016

Why randomization? Bias Motivating example: Smoking & FEV Relationship between the precision of unadjusted and adjusted Statistical role of variables Adjusted vs. unadjusted estimates effects Precision of adjusted estimators I Consider the following linear regression models: Nonadaptive Randomization Complete randomization Blocked randomization 1. Unadjusted model: E[Yi ]=0 + 1Xi Stratified randomization Adaptive Randomization Covariate adaptive 2. Adjusted model: E[Yi ]=0 + 1Xi + 2Wi randomization Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 :25

Precision of Estimators SISCR UW - 2016 Relationship between the precision of unadjusted and adjusted

estimates Why randomization? Bias Motivating example: I Proposition 2: Smoking & FEV 1. For the unadjusted model, Statistical role of variables Adjusted vs. unadjusted effects 2 Precision of adjusted Y X estimators Var[ˆ1]= | nvar(X) Nonadaptive Randomization Complete randomization 2. For the adjusted model, Blocked randomization Stratified randomization 2 Adaptive Y X,W Randomization Var[ˆ1]= | 2 Covariate adaptive nvar(X)(1 rXW ) randomization Response adaptive 2 2 2 randomization where Y X,W = Y X 2 var(W X) | | | Logistics of Randomization 2 2 I Hence, if 2 = 0 then Y X,W < Y X 6 | |

SISCR - RCT, Day 2 - 3 :26 Statistical role of variables SISCR UW - 2016

Why randomization? Bias Implications of Propositions 1 & 2 (generalizeable to p coviarate Motivating example: Smoking & FEV case) Statistical role of variables Adjusted vs. unadjusted effects I Case 1: rXW = 0(X and W uncorrelated) and 2 = 0(W Precision of adjusted and Y unrelated) estimators Nonadaptive Randomization ˆ Complete randomization I From Proposition 1, 1 unbiased for 1 Blocked randomization Stratified randomization ˆ Adaptive I From Proposition 2, Var[1]=Var[ˆ1] Randomization Covariate adaptive randomization I Conclusion: Lose 1 degree of freedom for hypothesis tests Response adaptive randomization and CIs if adjusting for W Logistics of Randomization

SISCR - RCT, Day 2 - 3 :27

Statistical role of variables SISCR UW - 2016

Why randomization? Bias Implications of Propositions 1 & 2 (generalizeable to p coviarate Motivating example: Smoking & FEV case) Statistical role of variables Adjusted vs. unadjusted effects I Case 2: rXW = 0(X and W correlated) and 2 = 0(W and Precision of adjusted Y unrelated)6 estimators Nonadaptive Randomization ˆ Complete randomization I From Proposition 1, 1 unbiased for 1 Blocked randomization Stratified randomization ˆ Adaptive I From Proposition 2, Var[1] < Var[ˆ1] Randomization Covariate adaptive randomization I Conclusion: Mathematically estimating the same quantity Response adaptive randomization but lose precision when adjusting for W (nuisance variable) Logistics of Randomization

SISCR - RCT, Day 2 - 3 :28 Statistical role of variables SISCR UW - 2016

Why randomization? Bias Implications of Propositions 1 & 2 (generalizeable to p coviarate Motivating example: Smoking & FEV case) Statistical role of variables Adjusted vs. unadjusted effects I Case 3: rXW = 0(X and W uncorrelated) and 2 = 0(W Precision of adjusted and Y related) 6 estimators Nonadaptive Randomization ˆ Complete randomization I From Proposition 1, 1 unbiased for 1 Blocked randomization Stratified randomization ˆ Adaptive I From Proposition 2, Var[1] > Var[ˆ1] Randomization Covariate adaptive randomization I Conclusion: Mathematically estimating the same quantity Response adaptive randomization but gain precision when adjusting for W (precision variable) Logistics of Randomization

SISCR - RCT, Day 2 - 3 :29

Statistical role of variables SISCR UW - 2016

Why randomization? Implications of Propositions 1 & 2 (generalizeable to p coviarate Bias Motivating example: case) Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Case 4: rXW = 0(X and W correlated) and 2 = 0(W and effects 6 6 Precision of adjusted Y related) estimators Nonadaptive Randomization I From Proposition 1, ˆ1 biased for 1 Complete randomization Blocked randomization Stratified randomization

I From Proposition 2, no definitive statement about the Adaptive Randomization Covariate adaptive randomization Response adaptive I Conclusion: W is a confounder and decision to adjust randomization Logistics of should be based on what you are trying to estimate. Randomization

SISCR - RCT, Day 2 - 3 :30 Statistical role of variables SISCR UW - 2016

Why randomization? Bias Motivating example: Smoking & FEV Why do we care? Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I The above results provide the fundamental motivation for estimators

Nonadaptive Randomization 1. The use and types of randomization (balance of Complete randomization confounders) Blocked randomization Stratified randomization

Adaptive 2. The consideration of analytic methods under various types Randomization Covariate adaptive of randomization randomization Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 :31

Methods of Randomization SISCR UW - 2016

Why randomization? Cause and Effect Bias Motivating example: Smoking & FEV I Necessary conditions for establishing cause and effect of Statistical role of variables Adjusted vs. unadjusted a treatment effects Precision of adjusted estimators 1. The treatment should precede the effect Nonadaptive Randomization Complete randomization Blocked randomization I Beware protopathic signs (eg. Marijuana and risk of MI within Stratified randomization 3 hours) Adaptive Randomization Covariate adaptive 2. When comparing groups differing in their treatment, the randomization Response adaptive groups should be comparable in every other way (at randomization baseline) (see previous discussion on confounding) Logistics of Randomization

SISCR - RCT, Day 2 - 3 :32 Methods of Randomization SISCR UW - 2016 Cause and Effect Randomization is the major way in which cause and effect is Why randomization? established Bias Motivating example: Smoking & FEV Statistical role of variables I Ensures comparability of populations Adjusted vs. unadjusted effects Precision of adjusted estimators I Each treatment group drawn from same population Nonadaptive I Differences in other prognostic factors will only differ by Randomization random Complete randomization Blocked randomization Stratified randomization

I Provides balance on the total effect of all other prognostic Adaptive Randomization factors Covariate adaptive I May not provide balance on each individual factor randomization Response adaptive randomization

Logistics of I Note: Sequential allocation of patients is not Randomization randomization

I Possible time trends in recruitment, treatments, etc.

SISCR - RCT, Day 2 - 3 :33

Nonadaptive Randomization SISCR UW - 2016 General statements on randomization

Why randomization? I Randomization is our friend... Bias Motivating example: Smoking & FEV I If we randomize, we do not (on average) need to worry Statistical role of variables Adjusted vs. unadjusted about differences between the treatment groups with effects respect to factors present at time of randomization Precision of adjusted estimators

Nonadaptive I Any difference in outcomes can be attributed to treatment Randomization Complete randomization I However, recognize that treatment can lead to differential use Blocked randomization of other ancillary treatments Stratified randomization Adaptive Randomization I But like all friends, we must treat it with respect. Covariate adaptive randomization Response adaptive randomization I We must analyze our data in groups defined at the time of Logistics of randomization Randomization

I Discarding or on randomized subjects may lead to bias (It certainly leads to diminished scientific credibility)

SISCR - RCT, Day 2 - 3 :34 Nonadaptive Randomization SISCR UW - 2016 Impact on data analysis

In presence of randomized treatment assignment Why randomization? Bias Motivating example: Smoking & FEV I Intent to treat analysis (ITT) Statistical role of variables Adjusted vs. unadjusted I Based on randomization effects Precision of adjusted estimators I Confounding not an issue (on average) Nonadaptive Randomization I P value measures probability of observed effects occurring Complete randomization Blocked randomization due only to randomization imbalance Stratified randomization

Adaptive Randomization I Gain precision if Covariate adaptive randomization I Control important predictors, or Response adaptive randomization I Adjust for stratification variables Logistics of Randomization

I Subgroup analyses

I If effect modification is concern I Pre-specification

SISCR - RCT, Day 2 - 3 :35

Nonadaptive Randomization SISCR UW - 2016 Randomization strategies

Why randomization? I Complete randomization (CRD) Bias Motivating example: Smoking & FEV Statistical role of variables I Blocked randomization Adjusted vs. unadjusted effects I Ensure balance after every k patients Precision of adjusted estimators I Ensure closer adherence to randomization ratio Nonadaptive I Undisclosed block sizes to prevent bias Randomization Complete randomization Blocked randomization Stratified randomization I Stratified randomization Adaptive I Separately within strata defined by strong risk factors Randomization Covariate adaptive I Lessens chance of randomization imbalance randomization Response adaptive I Need to consider how many variables can be used randomization Logistics of Randomization I Dynamic randomization

I Adaptive randomization to achieve best balance on marginal distribution of covariates

SISCR - RCT, Day 2 - 3 :36 Nonadaptive Randomization SISCR UW - 2016

Complete randomization Why randomization? Bias I The simplest form of randomization is independent Motivating example: Smoking & FEV randomization of each individual Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I With each accrued subject a (possibly biased) coin is estimators tossed to determine which arm Nonadaptive Randomization Complete randomization Blocked randomization I Probability of treatment arm = r/(r + 1) Stratified randomization

I Independence of successive randomizations Adaptive Randomization Covariate adaptive randomization I Possible issues with complete randomization include Response adaptive randomization

Logistics of I Bias, Randomization I Face validity, and I Precision

SISCR - RCT, Day 2 - 3 :37

Nonadaptive Randomization SISCR UW - 2016 Complete randomization

I On average (across repeated ) Why randomization? Bias Motivating example: Smoking & FEV I No correlation between treatment variable and other Statistical role of variables covariates Adjusted vs. unadjusted effects Precision of adjusted estimators I Individual type I errors come from samples in which other Nonadaptive covariates are imbalanced Randomization Complete randomization Blocked randomization Stratified randomization cov(X, W ) Adaptive ˆ Randomization E[1]=1 + 2 Covariate adaptive var(X) randomization Response adaptive randomization

Logistics of var(W ) Randomization = 1 + rXW 2 s var(X)

SISCR - RCT, Day 2 - 3 :38 Lecture 5: Randomization Methods April 21, 2010

Randomization Strategies Complete Randomization (CRD)

•Complete randomization (CRD) •With each accrued subject a (possibly biased) coin is •Blocked randomization tossed to determine which arm –Ensure balance after every k patients –Probability of treatment arm = r / (r + 1) –Ensure closer adherence to randomization ratio –Independence of successive randomizations –Undisclosed block sizes to prevent bias •Stratified randomization •Issues –Separately within strata defined by strong risk factors –Bias •Lessens chance of randomization imbalance –Face validity – Need to consider how many variables can be used –Precision •Dynamic randomization –Adaptive randomization to achieve best balance on Nonadaptive Randomization SISCR 17 18 marginal distribution of covariates UW - 2016

Complete randomization Why randomization? Bias I Typical to consider face validity of randomization in a Motivating example: Smoking & FEV “Table 1" Statistical role of variables CRD: Unbiased Face Validity: Table 1 Adjusted vs. unadjusted effects Precision of adjusted •On average (across repeated experiments) Methotrexate Arm Placebo Arm estimators n Mean (SD; Min – Max) n Mean (SD; Min – Max) Nonadaptive –No correlation between treatment variable and other Randomization Age (yrs) 132 50.4 (8.5; 32 - 69) 133 52.2 (8.5; 26 - 67) covariates Complete randomization Female 132 92.4% 133 92.5% Blocked randomization –Individual type I errors come from samples in which Pruritus score 116 7.7 (3.8; 4 - 16) 124 6.9 (3.8; 4 - 20) Stratified randomization other covariates are imbalanced Splenomegaly 131 8.4% 133 10.5% Adaptive Randomization Telangiectasia 132 4.6% 133 11.3% Covariate adaptive Edema 132 6.1% 133 3.0% W randomization Alkaline phosphatase 132 242.6 (145.9; 53 - 933) 133 245.0 (187.6; 66 - 1130) Response adaptive 1 1 XW 2 randomization X ALT 131 54.5 (41.7; 12 - 202) 132 50.6 (41.4; 12 - 311) Logistics of Total bilirubin 132 0.7 (0.4; 0.1 - 2.7) 133 0.7 (0.4; 0.1 - 2.4) Randomization Albumin 132 4.0 (0.3; 3.1 - 6.0) 133 4.0 (0.3; 3.0 - 4.8) Prothrombin time INR 124 1.0 (0.1; 0.7 - 1.3) 132 1.0 (0.1; 0.7 - 1.3) Mayo score 128 3.8 (0.8; 1.6 - 6.3) 133 3.9 (0.8; 1.6 - 6.1) Avg stage 128 2.2 (0.9; 1.0 - 4.0) 128 2.3 (0.9; 1.0 - 4.0) 19 20 Avg fibrosis 128 1.2 (0.8; 0.0 - 3.0) 128 1.3 (0.9; 0.0 - 3.0)

SISCR - RCT, Day 2 - 3 :39

Design of Medical Studies, SPR 2010 5

Nonadaptive Randomization SISCR Complete6/11/2014 randomization PubMed Central, Table 1: West J Med. May 2002; 176(3): 192–194. UW - 2016

I Consider differences in baseline stoke severity in a multi-center randomized comparing tissue Why randomization? Bias plasminogen activator (tPA) for the treatment of acute Motivating example: Smoking & FEV ischemic stroke Statistical role of variables PMC full text: West J Med. May 2002; 176(3): 192–194. Adjusted vs. unadjusted I effects PercentageCopyright/ ofLice patientsnse ► Re (Nques =t pe 320)rmission into re theuse 91 to 180-minute Precision of adjusted subgroups with a specific baseline National Institutes of estimators TabHealthle 1 Stroke Scale (NIHSS) score (Marler et al., Nonadaptive Randomization PercNeurologyentage of patients, ( 2000)N = 320) in the 91 to 180-minute subgroups witha specific baseline National Institutes oCompletef randomization * Health Stroke Scale (NIHSS)score Blocked randomization Stratified randomization

Baseline NIHSS score tPA-treated patients, % (n = 153) Patients given placebo, % (n = 167) Adaptive Randomization 0-5 19.0 4.2 Covariate adaptive randomization 6-10 24.2 27.5 Response adaptive 11-15 17.0 21.0 randomization 16-20 21.6 19.8 Logistics of Randomization >20 18.3 27.5 tPA = tissue plasminogen activator

*From Marler etal.2 “The marked imbalance in baseline stroke severity in the 91 to 180-minute groups of the NINDS trial suggests that the NINDS trial lacks internal validity." -Mann, West J. of Med (2002) SISCR - RCT, Day 2 - 3 :40

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1071714/table/tbl1/ 1/1 Lecture 5: Randomization Methods April 21, 2010

Nonadaptive Randomization SISCR UW - 2016 Complete randomization CRD: Face Validity CRD: Face Validity I Table 1: Potential for imbalance in covariates Why randomization? Bias Motivating example: Smoking & FEV •TableI Depends 1: Potential on number for imbalance of covariates in and covariates correlations among Statistical role of variables •Of course, statistical significance is not the issue Adjusted vs. unadjusted –Dependsthem on number of covariates and correlations effects –“Conditional confounding” I Probability of at least one “significant" imbalance Precision of adjusted among them estimators •How does unadjusted estimate compare to adjusted Nonadaptive –Probability of at least one “significant”imbalance Randomization estimate? Number Worst Correlation Complete randomization Blocked randomization •Product of sample correlation between X and W and Displayed Case 0.00 0.30 0.50 0.75 0.90 Stratified randomization adjusted association between Y and W Adaptive Randomization 1 .050 .050 .050 .050 .050 .050 Covariate adaptive randomization W 2 .100 .098 .095 .090 .081 .070 Response adaptive r randomization 1 1 XW 2

3 .150 .143 .137 .126 .104 .084 Logistics of X 5 .250 .226 .208 .184 .138 .101 Randomization 10 .500 .401 .353 .284 .193 .127 20 1.000 .642 .540 .420 .258 .154 21 22 50 1.000 .923 .806 .624 .353 .193

SISCR - RCT, Day 2 - 3 :41

NonadaptiveDemonstration Randomization of the Problem SISCR Low Association: 2-sided Complete randomization UW - 2016

I •ConsiderOf course, statisticala CRD in significancepresence of is not the issue •ROC curve for covariate imbalance “explaining”statistical –4 highly correlated predictors with larger importance Why randomization? significance under the null Bias Motivating example: I The–6 real independent concern is predictors “conditional with confounding" smaller importance Smoking & FEV n100b0.1r0.5 Two-sided (Rsqr Maj 0.122 Full 0.223 ) Statistical role of variables – No treatment effect Adjusted vs. unadjusted effects I How does unadjusted estimate compare to adjusted Precision of adjusted estimate? estimators Nonadaptive •I Questions about unadjusted analysis Product of sample correlation between X (treatment) and W Randomization –(potential What is type confounder) I error? and 0.025 adjusted association between Y Complete randomization Blocked randomization

–What(outcome) does and imbalanceW in predictors tell us about type I Stratified randomization Sensitivity Adaptive error? Randomization cov(X, W ) Covariate adaptive •Sensitivity,ˆ specificity of imbalance in predictors randomization E[1]=1 + 2 Response adaptive under null hypothesisvar(X) randomization 0.0 0.2 0.4 0.6 0.8 1.0

2 Logistics of 0.0 0.2 0.4 0.6 0.8 1.0 •Dependence on R of measured covariates 23 Randomization 24 1 - Specificity var(W ) = 1 + rXW 2 s var(X)

Design of Medical Studies, SPR 2010 SISCR - RCT, Day 2 - 3 :42 6 Nonadaptive Randomization SISCR Complete randomization UW - 2016

I Spurious results due to covariate imbalance Why randomization? Bias I Unconditionally: Unbiased so no problem Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I CONSORT Item 15 : “Although proper effects prevents selection bias, it does not guarantee that groups are Precision of adjusted estimators equivalent at baseline. Any differences in baseline Nonadaptive characteristics are, however, the result of chance rather than Randomization bias." Complete randomization Blocked randomization Stratified randomization

I Conditional on obtained randomization: Adaptive Randomization Covariate adaptive randomization I IF covariates are strongly predictive of outcome, then Response adaptive covariate imbalance increases type I error randomization I But need to consider that combined effect of other measured Logistics of and unmeasured covariates may provide balance Randomization

I Ultimately, however, we need to have credible results

I We do not always get to choose what others believe

SISCR - RCT, Day 2 - 3 :43

Nonadaptive Randomization SISCR UW - 2016

Why randomization? Bias Motivating example: Smoking & FEV Precision Statistical role of variables Adjusted vs. unadjusted effects I Impact of completely randomized design on precision of Precision of adjusted inference estimators Nonadaptive Randomization I Impact of imbalance in sample sizes Complete randomization Blocked randomization I The number accrued to each arm is random Stratified randomization Adaptive Randomization I Impact of imbalance in covariates Covariate adaptive randomization I “One statistician’s mean is another statistician’s variance" Response adaptive randomization

Logistics of Randomization

SISCR - RCT, Day 2 - 3 :44 Nonadaptive Randomization SISCR UW - 2016 Randomization ratio

Why randomization? I Most efficient Bias Motivating example: I When test statistics involve a sum, choose ratio equal to Smoking & FEV ratio of standard deviations Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators I Most ethical for patients on study Nonadaptive I Assign more patients to best treatment Randomization Complete randomization I Many sponsors / patients presume new treatment Blocked randomization I (Adaptive randomization: Play the winner) Stratified randomization Adaptive Randomization I Most ethical for general patient population Covariate adaptive randomization I Response adaptive Whatever is most efficient (generally not adaptive) randomization

Logistics of Randomization I Other goals

I Attaining sufficient patients exposed to new treatment I Maintaining DSMB blind

SISCR - RCT, Day 2 - 3 :45

Nonadaptive Randomization SISCR UW - 2016 Randomization ratio : Optimal r (fixed n)

I Suppose we are constrained by maximal sample size Why randomization? Bias n = n1 + n2 Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Smallest when effects Precision of adjusted estimators

Nonadaptive n1 s1 Randomization r = = Complete randomization n2 s2 Blocked randomization Stratified randomization

Adaptive where si is the of response in group i, Randomization Covariate adaptive randomization i = 1, 2 Response adaptive randomization

Logistics of Randomization I When we are unconstrained by maximal sample size we still hit a point of diminishing returns

I Often quoted: r = 5 I Really depends on ratio of standard deviations...

SISCR - RCT, Day 2 - 3 :46 Nonadaptive Randomization SISCR Randomization ratio : Optimal r (fixed n) UW - 2016

Optimal Sample Size Ratio for Fixed n1 + n2 Why randomization? Bias Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted 3.0 effects Precision of adjusted s1=3 x s2 estimators

s1=2 x s2 Nonadaptive 2.5 Randomization s1=1 x s2 Complete randomization Blocked randomization Stratified randomization X 2.0 r=3 Adaptive Randomization

Standard Error Covariate adaptive randomization X 1.5 Response adaptive r=2 randomization

Logistics of Randomization X 1.0 r=1

0 5 10 15 20 25

Sample Size Ratio (r = n1/n2)

SISCR - RCT, Day 2 - 3 :47

Nonadaptive Randomization SISCR Randomization ratio : Diminishing returns UW - 2016

Diminishing Returns : r > 5? Why randomization? Bias Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted 3.0 s1=3 x s2 effects Precision of adjusted s1=2 x s2 estimators s1=1 x s2 Nonadaptive 2.5 Randomization Complete randomization Blocked randomization Stratified randomization 2.0

Adaptive Randomization

Standard Error Covariate adaptive

1.5 randomization Response adaptive X randomization Logistics of 1.0 X Randomization X 0.5 0 5 10 15 20

Sample Size Ratio (r = n1/n2)

SISCR - RCT, Day 2 - 3 :48 Lecture 5: Randomization Methods April 21, 2010

Comment: Diminishing Returns Comment: Diminishing Returns

Diminishing Returns: r > 5? •When we are unconstrained by maximal sample size we still hit a point of diminishing returns –Often quoted: r = 5 Standard Error Standard Error

XX 1.0 1.5 2.0 2.5 3.0 1.0 1.5 2.0XX 2.5 3.0 s1s1 == 33 ** s2s2 XX s1s1 == 22 ** s2s2 s1s1 == s2s2 05101520250510152025 Nonadaptive Randomization SampleSample SizeSize RatioRatio rr == n1n1 // n2n2 33 34 SISCR UW - 2016 Complete randomization

I It is possible, in smaller studies, that a completely Why randomization? randomized design with high randomization ratio may not Bias Motivating example: randomize at least two subjects to each arm Smoking & FEV Statistical role of variables CRD: Loss from Wrong Ratio CRD: Probability 0 or 1 on an Arm Adjusted vs. unadjusted effects Precision of adjusted I Consider the probability that a CRD may not randomize at estimators •CRD may not attain optimal ratio least•CRD two subjects may not to randomize each arm as at aleast function two ofsubjects the total per Nonadaptive Randomization – Following table explores practical inefficiency trial sizearm and randomization ratio Complete randomization Blocked randomization –(True inefficiency is infinite due to possibility of no Stratified randomization subjects randomized to one group) Adaptive N r= 1 r= 2 r= 3 r= 5 r=10 Randomization Covariate adaptive 20 0.0000 0.0033 0.0243 0.1304 0.4459 randomization N r= 1 r= 2 r= 3 r= 5 r=10 Response adaptive randomization 20 1.0599 1.0652 1.0694 *** *** 50 0.0000 0.0000 0.0000 0.0012 0.0511 Logistics of 50 1.0213 1.0219 1.0229 1.0258 1.0282 100 0.0000 0.0000 0.0000 0.0000 0.0008 Randomization 100 1.0103 1.0104 1.0106 1.0111 1.0130 200 0.0000 0.0000 0.0000 0.0000 0.0000 200 1.0051 1.0051 1.0051 1.0053 1.0056 500 0.0000 0.0000 0.0000 0.0000 0.0000 500 1.0020 1.0020 1.0020 1.0020 1.0021 1000 0.0000 0.0000 0.0000 0.0000 0.0000 1000 1.0010 1.0010 1.0010 1.0010 1.0010 35 36 SISCR - RCT, Day 2 - 3 :49

Nonadaptive Randomization SISCR Design of Medical Studies, SPR 2010 9 UW - 2016

Efficiency loss from imbalance Why randomization? Bias Motivating example: I Covariates may be imbalanced across arms Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I Variability across replicated experiments increased if Precision of adjusted estimators

important predictor not controlled Nonadaptive Randomization Complete randomization I Recall Blocked randomization Stratified randomization

Adaptive 2 Randomization ˆ Y X Covariate adaptive Var[1]= | randomization nvar(X) Response adaptive randomization with Logistics of 2 2 2 Randomization Y X = 2 var(W X)+Y X,W | | |

SISCR - RCT, Day 2 - 3 :50 Nonadaptive Randomization SISCR How to improve performance? UW - 2016

I If we adjust for important covariates, we will often gain Why randomization? precision Bias Motivating example: Smoking & FEV Statistical role of variables I Face validity in Table 1 if readers recognize that adjustment Adjusted vs. unadjusted accounts for any observed imbalance effects Precision of adjusted estimators

Nonadaptive I Caveats: Randomization Complete randomization Blocked randomization I If covariate imbalance by arm, model misspecification can Stratified randomization Adaptive be an issue regarding conditional bias Randomization Covariate adaptive randomization I If covariate imbalance by arm, lack of effect can be an issue Response adaptive regarding variance inflation randomization Logistics of Randomization I If adjustment not TOTALLY prespecified, “intent to cheat" analysis can be an issue

I Loss of precision from imperfect model should not be too much of an issue in most situations

SISCR - RCT, Day 2 - 3 :51

Nonadaptive Randomization SISCR UW - 2016

Issues with complete randomization Why randomization? Bias I Imbalance across arms in sample sizes Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Not much of an issue with large sample sizes effects I Precision of adjusted Could be problematic with sequential sampling estimators

I Interim analyses of data early in the study Nonadaptive Randomization Complete randomization I Imbalance across arms in time trends Blocked randomization Stratified randomization

Adaptive Randomization I Outcome may be associated with time of accrual Covariate adaptive randomization Response adaptive randomization I is sometimes used to ensure Logistics of Randomization

I Proper ratio of sample sizes across groups, and I Balance across arms over time

SISCR - RCT, Day 2 - 3 :52 Nonadaptive Randomization SISCR UW - 2016 Mechanisms leading to time trends

Why randomization? I Patients accrued early may differ from those accrued later, Bias Motivating example: because Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I Backlog of eligible patients Precision of adjusted estimators

Nonadaptive I Startup of new clinical sites Randomization Complete randomization Blocked randomization I Pressure to increase accrual Stratified randomization

Adaptive I Secular trends in beliefs about intervention Randomization Covariate adaptive randomization Response adaptive I (Made much worse if any interim results leak out) randomization Logistics of Randomization I Secular trends in diagnostic tools used for eligibility

I Secular trends in ancillary treatments

SISCR - RCT, Day 2 - 3 :53

Nonadaptive Randomization SISCR UW - 2016

Mechanisms leading to time trends Why randomization? Bias I Within every sequence of k patients, the ratio of treatment Motivating example: Smoking & FEV to control is exactly r : 1 Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I Within each “block" ordering of treatments is random estimators

Nonadaptive Randomization I Important caveats: Complete randomization Blocked randomization Stratified randomization

I Investigators must not know block size Adaptive Randomization Covariate adaptive randomization I Otherwise, decisions to enroll patients might be affected by Response adaptive knowledge of next assignment randomization Logistics of Randomization I Hence, often use “concealed blocks of varying sizes" (often termed a “random block design")

SISCR - RCT, Day 2 - 3 :54 Nonadaptive Randomization SISCR UW - 2016

Alternative strategy : Urn Model Why randomization? Bias Motivating example: 1. Begin with k white balls and r k black balls in an urn Smoking & FEV ⇥ Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted 2. Upon accrual of a patient draw a ball from urn estimators

Nonadaptive Randomization I White control; black treatment ! ! Complete randomization I After every white ball withdrawn, return 1 white ball and Blocked randomization Stratified randomization r m black balls ⇥ Adaptive I After every r-th black ball withdrawn, return r black balls and Randomization Covariate adaptive m white balls randomization Response adaptive randomization

Logistics of I Such a strategy tends to behave like small blocks early Randomization and complete randomization later, depending on k and m

Lecture 5: Randomization Methods April 21, 2010

SISCR - RCT, Day 2 - 3 :55

Nonadaptive Randomization SISCR Alternative Strategy: Urn Model Comparison of Blocking Strategies UW - 2016 Comparison of blocking strategies

I SD proportion on treatment for 3:1 randomization Why randomization? •Begin with k white balls and rk black balls in an urn •SD proportion on treatment for 3:1 randomization Bias I Urn (k = 1, m = 1) vs Blocking (size = 8) vs CRD Motivating example: – Urn (k=1, m=1) vs Blocking (size = 8) vs CRD Smoking & FEV Statistical role of variables Adjusted vs. unadjusted •Upon accrual of a patient draw a ball from urn effects Precision of adjusted –White control; black treatment estimators Nonadaptive –After every white ball withdrawn, return 1white ball and Randomization Complete randomization rm black balls Blocked randomization Stratified randomization

–After every r-th black ball withdrawn, return r black balls Adaptive Randomization and m white balls Covariate adaptive randomization Response adaptive randomization SD Proportion on Treatment on Proportion SD Logistics of •Such a strategy tends to behave like small blocks early and CRD Randomization complete randomization later, depending on k and m Urn

0.0 0.1Block 0.2 0.3 0.4 0.5 45 0100200300400 46 Number Accrued

SISCR - RCT, Day 2 - 3 :56

Statistical Inference Nonadaptive Randomization •Impact on relative to CRD –Bias properties unchanged –Face validity largely unchanged Stratified Randomization •We rarely report accrual patterns over time –Precision slightly improved due to achieving closer to Where am I going? desired randomization ratio

–Precision could be improved if adjust for blocks as a •Stratified randomization is sometimes used to ensure proper ratio of random effect in analysis sample sizes across subgroups defined by important covariates, thereby •This is rarely done, except in re-randomization test –Decreasing conditional bias, – Improving face validity, and –Large number of small blocks, often with small variance of –Possibly improving precision the random effects •Major improvements in precision are gained only with adjustment for important stratification variables 47 48

Design of Medical Studies, SPR 2010 12 Nonadaptive Randomization SISCR UW - 2016

Statistical inference after blocking Why randomization? Bias I Impact on statistical inference relative to CRD Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Bias properties unchanged effects Precision of adjusted estimators I Face validity largely unchanged Nonadaptive Randomization Complete randomization I We rarely report accrual patterns over time Blocked randomization Stratified randomization

Adaptive I Precision slightly improved due to achieving closer to Randomization Covariate adaptive desired randomization ratio randomization Response adaptive randomization I Precision could be improved if adjust for blocks as a random Logistics of effect in analysis Randomization

I This is rarely done, except in re-randomization test

SISCR - RCT, Day 2 - 3 :57

Nonadaptive Randomization SISCR UW - 2016 Issues with complete randomization

I Imbalance across arms in covariate distribution Why randomization? Bias Motivating example: I Loss of face validity Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I Conditional bias Precision of adjusted estimators

Nonadaptive I Not much of an issue with large sample sizes Randomization Complete randomization Blocked randomization I Could be problematic with sequential sampling Stratified randomization

I Interim analyses of data early in the study Adaptive Randomization Covariate adaptive randomization I Could be problematic with subgroup analyses Response adaptive I Possibility of very inefficient randomization ratio in small randomization subgroups Logistics of Randomization

I Stratified randomization is often used to ensure proper ratio of sample sizes across subgroups defined by important covariates

SISCR - RCT, Day 2 - 3 :58 Nonadaptive Randomization SISCR UW - 2016

Stratified randomization Why randomization? Bias Motivating example: I Strata are defined based on values of important covariates Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I E.g., sex, age, disease severity, clinical site Precision of adjusted estimators

Nonadaptive Randomization I Within each stratum defined by a unique combination of Complete randomization stratification variables, CRD or blocked randomization Blocked randomization Stratified randomization

Adaptive Randomization I Important caveats: Covariate adaptive randomization Response adaptive I Number of strata is exponential in number of stratification randomization variables Logistics of Randomization I E.g., 4 two level stratification variables 16 strata )

SISCR - RCT, Day 2 - 3 :59

Nonadaptive Randomization SISCR UW - 2016

Statistical inference Why randomization? I Impact on statistical inference relative to CRD Bias Motivating example: Smoking & FEV Statistical role of variables I Bias properties unchanged Adjusted vs. unadjusted effects Precision of adjusted estimators I Face validity improved for most important variables Nonadaptive Randomization Complete randomization I Precision improved due to achieving closer to desired Blocked randomization randomization ratio Stratified randomization Adaptive Randomization I Precision could be further improved if adjust for stratification Covariate adaptive variables in analysis randomization Response adaptive randomization

I This should be done! (Without adjustment for strata, may Logistics of Randomization even lose power for some alternatives) I Requires pre-specification of analysis model to avoid “intent to cheat" analysis

SISCR - RCT, Day 2 - 3 :60 Nonadaptive Randomization SISCR UW - 2016

Additional advantages of stratified randomization Why randomization? Bias Motivating example: I Additional advantages of stratification Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Balance within clinical center effects Precision of adjusted I Especially if issues estimators Nonadaptive Randomization I Balance for interim analyse Complete randomization Blocked randomization Stratified randomization I Balance for subgroup analyses Adaptive Randomization Covariate adaptive randomization I Also, stratified randomization does not preclude the use of Response adaptive blocking randomization Logistics of Randomization

I Common to combine the two...blocking within strata

SISCR - RCT, Day 2 - 3 :61

Adaptive Randomization SISCR UW - 2016 Issues with stratified randomization

Why randomization? I The need to stratify on all combinations of variables Bias Motivating example: Smoking & FEV I Good news: Statistical role of variables Adjusted vs. unadjusted I Balances on interactions as well as main effects effects Precision of adjusted estimators

I Bad news: Nonadaptive Randomization I Effect of interactions might be quite small Complete randomization I Really only need to adjust on “counterfactual" outcome based Blocked randomization on linear combination of all covariates Stratified randomization Adaptive Randomization Covariate adaptive I Stratified randomizations has drawbacks in the presence randomization Response adaptive of sparse data randomization Logistics of Randomization I Because of this, some authors have described dynamic randomization processes that will allow balancing on more covariates

SISCR - RCT, Day 2 - 3 :62 Adaptive Randomization SISCR UW - 2016 Dynamic randomization

Why randomization? I Subjects are assigned to the treatment arm that will Bias Motivating example: achieve best balance Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I “Minimization": minimize the difference between the Precision of adjusted distribution of covariate effects between arms estimators I Define a “distance" between arms for covariate vectors Nonadaptive Randomization I Probability of assignment depends upon arm that would Complete randomization provide smallest difference Blocked randomization Stratified randomization

Adaptive I Two arms are “distant" based on one of: Randomization Covariate adaptive randomization Response adaptive I Randomization ratio very different from r : 1 in some stratum randomization I Summary measure of distribution of (Wi1,...,Wip) differs Logistics of Randomization I Mean, median, variance, ...

I Distribution of (Wi1,...,Wip) differs I Contribution of covariates to the outcome differs

SISCR - RCT, Day 2 - 3 :63

Adaptive Randomization SISCR UW - 2016

Conditional confounding Why randomization? Bias Motivating example: I Consider unadjusted and adjusted (linear) models for an Smoking & FEV outcome Y : Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted estimators 1. Unadjusted model: E[Yi ]=0 + 1Xi Nonadaptive Randomization ~ T ~ Complete randomization 2. Adjusted model: E[Yi ]=0 + 1Xi + W i Blocked randomization Stratified randomization or in matrix notation Adaptive Randomization Covariate adaptive randomization 1. Unadjusted model: E[Y~ ]=X~ Response adaptive randomization

Logistics of 2. Adjusted model: E[Y~ ]=X~ + W~ Randomization

SISCR - RCT, Day 2 - 3 :64 Adaptive Randomization SISCR UW - 2016

Conditional confounding Why randomization? Bias Motivating example: I Then it can be shown that Smoking & FEV Statistical role of variables Adjusted vs. unadjusted ~ T 1 T ~ effects E[]=~ +(X X) X W Precision of adjusted estimators

Nonadaptive I This implies that b Randomization Complete randomization p Blocked randomization ¯ ¯ Stratified randomization 1 = 1 + W1j W0j j · · Adaptive j=1 Randomization Covariate adaptive X randomization Response adaptive with randomization n 1 Logistics of ¯ Randomization Wkj = Wij 1[Xi =k] · nk Xi=1

SISCR - RCT, Day 2 - 3 :65

Adaptive Randomization SISCR UW - 2016 This provides reasonable ways to define distance metrics

I Based on contribution to confounding : Why randomization? Bias Motivating example: p Smoking & FEV Statistical role of variables ~ ¯ ¯ Adjusted vs. unadjusted d(X, W)= W1j W0j j effects · · j=1 Precision of adjusted X estimators Nonadaptive I Weighted distance between standardized means : Randomization Complete randomization Blocked randomization p ¯ ¯ Stratified randomization W1j W0j ~ · · Adaptive d(X, W)= cj Randomization SD(Wj j=1 Covariate adaptive X randomization Response adaptive randomization I Weighted imbalance in n across strata ⌦ ,..., ⌦s : 1 Logistics of Randomization S n n ~ d(X, W)= cs 1[Xi =1]1[W~ ⌦ ] 1[Xi =0]1[W~ ⌦ ] i 2 s i 2 s s=1 i=1 i=1 X X X SISCR - RCT, Day 2 - 3 :66 Adaptive Randomization SISCR UW - 2016

Conditional confounding Why randomization? Bias I Spurious associations will be minimized if means of Motivating example: Smoking & FEV important predictors are balanced across treatment arms Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I The greater the value of j the more important it is for the estimators

means of the j-th covariate to be equal Nonadaptive I (Presumes linear model reasonable approximation) Randomization Complete randomization Blocked randomization Stratified randomization I We could use estimates of the of j ’s to define the distance Adaptive between the arms (or just balance means) Randomization Covariate adaptive randomization Response adaptive I Balancing group sizes across covariates will tend to have randomization means balanced by randomization Logistics of Randomization

I Group sizes within strata may matter for subgroup analyses

SISCR - RCT, Day 2 - 3 :67

Adaptive Randomization SISCR UW - 2016 Dynamic randomization

I Subjects are assigned to the treatment arm that will Why randomization? Bias achieve best balance Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I When i-th patient accrued, compute a randomization effects Precision of adjusted probability, ⇡i , where estimators Nonadaptive ~ ~ Randomization i = d(X, W Xi = 1) d(X, W Xi = 0) Complete randomization | | Blocked randomization and Stratified randomization Adaptive Randomization Covariate adaptive ⇡i = Pr[Xi = 1]=f (i ), randomization Response adaptive with randomization Logistics of Randomization

I 0 ⇡ 1  i  I f (i ) monotonically decreasing in ⇡i I (generally seek to avoid ⇡i = 0 and ⇡i = 1)

SISCR - RCT, Day 2 - 3 :68 Adaptive Randomization SISCR UW - 2016

Inference : Population model Why randomization? Bias I Impact on statistical inference relative to CRD Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Bias properties unchanged effects Precision of adjusted estimators

I Face validity improved for most important variables Nonadaptive Randomization Complete randomization I Precision improved due to achieving closer to desired Blocked randomization randomization ratio Stratified randomization Adaptive Randomization I Covariate adaptive Precision could be further improved if adjust for stratification randomization variables in analysis for population model Response adaptive randomization

Logistics of I This should be done Randomization I Requires pre-specification of analysis model to avoid “intent to cheat" analysis

SISCR - RCT, Day 2 - 3 :69

Adaptive Randomization SISCR UW - 2016 Advantages and disadvantages

I Advantages: Why randomization? Bias Motivating example: I Typically improved face validity Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I Can handle an arbitrary number of covariates Precision of adjusted estimators I Depending on distance metric Nonadaptive Randomization Complete randomization I Disadvantages: Blocked randomization Stratified randomization

Adaptive I Logistically more involved Randomization Covariate adaptive randomization I Decreased credibility if too deterministic Response adaptive randomization I Approaches sequential allocation Logistics of I Some analytic strategies more complex (permutation tests Randomization for strong null)

I Does not necessarily facilitate subgroup analyses I Unless distance metric chosen carefully

SISCR - RCT, Day 2 - 3 :70 Adaptive Randomization SISCR UW - 2016 Ethics

Why randomization? I Clinical trials are experiments in human volunteers Bias Motivating example: Smoking & FEV I Individual ethics: Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I Patients on trial: Avoid continued administration of inferior estimators treatment Nonadaptive Randomization I Patients not yet on trial: Avoid starting inferior treatment Complete randomization Blocked randomization Stratified randomization I Group ethics: Adaptive Randomization Covariate adaptive I Facilitate rapid adoption of new beneficial treatments randomization I Avoid prolonging study of ineffective treatments Response adaptive randomization

Logistics of Randomization I Some authors have described dynamic randomization processes that attempt to minimize exposure of patients to harmful treatments

SISCR - RCT, Day 2 - 3 :71

Adaptive Randomization SISCR UW - 2016

Why randomization? Proposed solutions Bias Motivating example: Smoking & FEV I Most commonly used Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted I Sequential sampling estimators

I Interim analyses of data Nonadaptive Randomization I Terminate trials when credible decisions can be made Complete randomization Blocked randomization Stratified randomization I Also proposed Adaptive Randomization Covariate adaptive I Response adaptive randomization randomization Response adaptive I Change randomization probabilities as evidence accumulates randomization that one treatment might be best Logistics of Randomization I “Play the winner"

SISCR - RCT, Day 2 - 3 :72 Adaptive Randomization SISCR UW - 2016

Play the winner : Urn model Why randomization? Bias Motivating example: 1. Begin with k white balls and k black balls in an urn Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects Precision of adjusted 2. Upon accrual of a patient draw a ball from urn estimators

Nonadaptive Randomization I White control; black treatment Complete randomization ! ! Blocked randomization Stratified randomization

3. Observe outcome Adaptive Randomization Covariate adaptive randomization I If outcome is good, return m + 1 balls of same color as Response adaptive withdrawn randomization Logistics of I If outcome is bad, return 1 ball of same color as withdrawn Randomization and m balls of opposite color

SISCR - RCT, Day 2 - 3 :73

Adaptive Randomization SISCR UW - 2016

Bayesian methods Why randomization? Bias Motivating example: Smoking & FEV I An explicit Bayesian approach to dynamic randomization Statistical role of variables bases the randomization ratio on the current posterior Adjusted vs. unadjusted effects probability that one treatment is superior Precision of adjusted estimators

Nonadaptive I Ultimately, that is based on the number Randomization Complete randomization of good outcomes on each treatment (in conjunction with a Blocked randomization probability model for the response and a prior distribution) Stratified randomization Adaptive Randomization Covariate adaptive I Advantage of using Bayesian posterior probability randomization Response adaptive randomization

I Can easily handle continuous outcomes Logistics of Randomization I Can easily handle continuous randomization probabilities

SISCR - RCT, Day 2 - 3 :74 Adaptive Randomization SISCR UW - 2016

Why randomization? Bias Analytic issues Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Treatment of successive patients is not independent of effects Precision of adjusted previous patients treatment and results estimators

Nonadaptive Randomization I Possible bias in accrual of future patients Complete randomization Blocked randomization Stratified randomization

I Conditionally biased estimates of treatment effect in arm Adaptive with lower sample sizes Randomization Covariate adaptive randomization Response adaptive randomization I Bad early results tend to preclude regression to mean Logistics of Randomization

SISCR - RCT, Day 2 - 3 :75

Adaptive Randomization SISCR UW - 2016 Response-Adaptive Randomization (Example)

Why randomization? ECMO: Extracorporeal Membrane Oxygenation in Bias neo-natal respiratory failure Motivating example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted I Persistent pulmonary hypertension results in right to left effects Precision of adjusted shunt through the foramen ovale or ductus arteriosus estimators causing hypoxemia. ECMO is used to maintain life until the Nonadaptive Randomization condition resolves. Complete randomization Blocked randomization Stratified randomization I Trial 1 (Play the winner absolutely): Pediatrics (1985) Adaptive 76:479-487 Randomization Covariate adaptive I First subject was randomized to conventional medical therapy randomization (CMT); the infant died. Response adaptive randomization I Second subject given ECMO; infant lived. Logistics of I Next 8 subjects given ECMO; all lived. Randomization I Result: 100% mortality with CMT 0% with ECMO RR = 0.

SISCR - RCT, Day 2 - 3 :76 Adaptive Randomization SISCR UW - 2016 Response-Adaptive Randomization (Example)

ECMO Example (con’t): Why randomization? Bias Motivating example: I Trial 2 (Play the winner with higher probability): Pediatrics Smoking & FEV Statistical role of variables (1989) 84(6):957-63 Adjusted vs. unadjusted effects I Randomize until the 4th CMT death, then treat remainder with Precision of adjusted best approach. estimators I 19 babies in first phase (4/10 die with CMT; 0/9 die with Nonadaptive ECMO). Randomization Complete randomization I 20 babies on ECMO in second phase (1 death). Blocked randomization I Result: Stratified randomization 40% (4/10) mortality with CMT; Adaptive Randomization 3% (1/29) with ECMO; Covariate adaptive RR = 0.086. randomization Response adaptive randomization

I Trial 3 (conventional RCT): Pediatrics (1998) 101(4):E1 Logistics of Randomization I Randomize 185 infants (92 to CMT, 93 to ECMO) I Result: 59% (54/92) mortality with CMT; 32% (30/93) with ECMO; RR = 0.55.

SISCR - RCT, Day 2 - 3 :77

Adaptive Randomization SISCR UW - 2016 Response-Adaptive Randomization (Example)

ECMO Example (con’t): Why randomization? Bias Motivating example: Implications of the ECMO example: Smoking & FEV Statistical role of variables Adjusted vs. unadjusted effects I ECMO looked better with response-adaptive randomization. Precision of adjusted I Response-adaptive designs were not accepted as adequate estimators justification for ECMO. Nonadaptive Randomization I Inadequate study designs can delay introduction of beneficial Complete randomization treatments or prolong use of inferior treatments. Blocked randomization Stratified randomization

Adaptive “In fact, in the ECMO trial, the patient who failed on Randomization Covariate adaptive treatment B had the most extreme values on no fewer than randomization Response adaptive four important covariates (Paneth & Wallenstein, 1985), randomization Logistics of and was clearly the sickest. In effect, the trial provides no Randomization information whatsoever regarding the treatment comparison. "

-Begg (1990)

SISCR - RCT, Day 2 - 3 :78 Adaptive Randomization SISCR UW - 2016

Why randomization? Response-Adaptive Randomization (Example) Bias Motivating example: Smoking & FEV Statistical role of variables I The ECMO experience has tempered enthusiasm for Adjusted vs. unadjusted effects randomized PTW Precision of adjusted estimators

Nonadaptive I This being said, there may be times were Randomization Complete randomization response-adaptive randomization will work, but Blocked randomization Stratified randomization

Adaptive I There needs to be a clear dilemma re individual ethics Randomization Covariate adaptive I There will tend to be decreased group ethics randomization Response adaptive I It takes a lot of planning in order to obtain results that will be randomization

sufficiently credible Logistics of Randomization

SISCR - RCT, Day 2 - 3 :79

Logistics of Randomization SISCR UW - 2016 Methods: Logistics of Randomization

I Where to perform randomization: Why randomization? Bias Motivating example: Smoking & FEV I Central randomization: Statistical role of variables Adjusted vs. unadjusted I Phone calls to the coordinating center. effects I Sequences can be determined at the start of the study Precision of adjusted estimators (except with adaptive randomization). Nonadaptive Randomization Complete randomization I Distributed randomization: Computer programs, envelopes, Blocked randomization or lists at pharmacies. Stratified randomization Adaptive Randomization I Important principles: Covariate adaptive randomization Response adaptive randomization

I Strong quality assurance must be in place to ensure proper Logistics of randomization. Randomization I Ensure adequate concealment/blinding. I Provide for emergency unblinding. I Exact randomization scheme must be known for analysis.

SISCR - RCT, Day 2 - 3 :80