university of copenhagen department of university of copenhagen department of biostatistics

Faculty of Health Sciences Contents

◮ Planning statistical analyses with missing .

◮ Missing data Missing data types. ◮ due to missing data or improper handling of these. Analysis of repeated measurements 2017 ◮ Analyzing data with missing values using multiple imputations, likelihood inference, or inverse probability weighting.

Julie Lyng Forman & Lene Theil Skovgaard ◮ Analysis of longitudinal studies with death or other Department of Biostatistics, University of Copenhagen intercurrent events.

Suggested reading FLW (2011) chapters 18 +19, lecture notes.

2 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Outline What is missing data?

What to worry about when you have missing data Most investigations are planned to be balanced but almost inevitably turn out to have intermittent missing values , or Missing data types patients who drop-out for some reason . . .

Simple methods for handling missing data ◮ Just by coincidence (sample lost or ruined).

Advanced methods for handling missing data ◮ The patient moved away (may be worrysome). ◮ The patient has recovered (worrying, i.e. carrying Missing data in population average models (binary data) information).

Death and other intercurrent events in longitudinal studies ◮ The patient is too ill to show up (very serious, i.e. carrying unretrievable information). Evaluation

3 / 60 4 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Missing data is trouble Planning statistical analyses with missing data

Missing data should be addressed already in the planning stage. ◮ It complicates statistical analysis. 1. What are the outcomes and explanatory variables? ◮ It may bias statistical results beyond repair. 2. What are the parameters of interest (the study objective)? ◮ It compromises the causal interpretation of treatment effect in randomized trials. 3. Which variables may have missing values?

◮ It reduces statistical power since information is lost. 4. What are the likely reasons they are missing?

5. What other factors (auxiliary variables) could be associated The best way to handle missing data would be to prevent it, with missingness? Are they also associated with the outcome? but this is often not possible . This helps us decide:

Missing data should always be recognized as a limitation. 6. What statistical methods should be used for analyses?

5 / 60 6 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

The missing data mechanism Example: CKD study from lecture 1

It is important to understand WHY data is missing.

Investigate: ◮ If possible, ask the patients or investigators! ◮ Make separate spaghettiplots for completers and drop-outs. ◮ Make a table comparing the distribution of covariates and other characteristics between the drop-outs and the completers.

Speculate: ◮ Think about what differences there might be e.g. between completers and drop-outs in terms of unmeasured outcomes and confounders. ◮ How could these affect the results of your analysis . More drop outs due to adverse events in Eplerenone group! 7 / 60 8 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Study objectives FDA recommendations for clinical trials

What parameter is the target of the study or trial? In your study protocol please include a section describing how you plan to address missing data. of actual data had they all been collected, e.g. We recommend missing data be avoided by continuing to collect ◮ Change in mean over time of the entire study population. (efficacy and safety) data even from subjects who prematurely ◮ Difference in between two populations at a given time. discontinue study drug. ◮ Difference in means between the initially randomized Our preference is that the primary analysis 1) include all data, not treatment groups regardless of what treatment subjects just data while adhering to study drug, and 2) for the limited actually received ( intention to treat principle ). missing data that do occur, it be represented by what their response likely would have been had it been measured. Mean of counterfactual data had they all been collected, e.g. Because missing data tend to be associated with treatment ◮ Difference in means that would have been found if all subjects adherence, it would not be appropriate to have an analysis that had completed their assigned treatment . uses information from those with data who adhered to treatment ◮ Difference in populations means if all had survived until end of to describe what happened to those without data who did not follow-up . adhere to treatment.

9 / 60 10 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Your own data Outline

Think about the data from your own research project. What to worry about when you have missing data

Missing data types ◮ Are any data missing?

◮ How many? Simple methods for handling missing data

◮ Do you know WHY? Advanced methods for handling missing data

◮ Are the observed outcomes representative of the population Missing data in population average models (binary data) you wanted to study, or different somehow?

◮ What exactly is your study objective, considering the missing Death and other intercurrent events in longitudinal studies data? Evaluation

11 / 60 12 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Types and patterns of missingness MCAR: Missing completely at random Missing data taxonomy MCAR Missing completely at random. Examples: Unrelated to the outcome and the covariates. ◮ Sample lost in the mail. Note: Special case of MAR. ◮ Some data too expensive/inconvenient to collect from the MAR Missing at random. whole sample, hence only collected for a random subsample. Missingness is conditionally independent of the missing values given the observed data. Statistical consequences ◮ NMAR Not missing at random. Reduced power due to reduced sample size . Missingness is not conditionally independent of the ◮ Data may end up being unbalanced (software problem - ?) missing values given the observed data. ◮ Otherwise benign .

Missing data patterns in longitudinal studies If missing data is MCAR, then the complete cases form a random Monotone Drops out and stays out. representative subsample from the original study population . Intermittent (aka non-monotone) Comes back later. 13 / 60 14 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Average curves Example of a missing mechanism MAR Low values are good (e.g. blood pressure): The average curve is representative of the whole population when ◮ When the patient learns he is doing well, he might decide he data is complete or missing data is MCAR. no longer needs to attend visits and staying away does not ◮ When missing data is MAR or NMAR it is likely biased . affect his outcome.

Spaghettiplots are always ok. Sample averages are biased. Mean estimates from LMM are ok . 15 / 60 16 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Example of a missing data mechanism NMAR MAR: Missing at random

Low values are bad (e.g. lung function): R1 denotes the response indicator (1=observed, 0=missing). / ◮ When the person gets sufficiently ill, he drops out of the X Y0 labour market ( healthy worker effect ). Simplified DAG  ' /' potential drop out Y1 R1 only after baseline.   ~ Y2 ◮ Missingness may depend on past observed outcomes and covariates included in the model, e.g. treatment . ◮ Missingness may not depend on current outcome neither directly nor by means of unmeasured confounders . ◮ Future outcome of interest (de facto or counterfactual) after drop out may not depend on missingness. Sample averages are biased . Mean estimates from LMM too. 17 / 60 18 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

MAR depends on the covariates Case: Missing data in calcium study

Assume a treatment-gender (or -gene, or . . . ): ◮ Positive effect in women. Negative effect in men. ◮ Men are overall more likely to drop out.

An average positive change is found in the population if gender is not included in the model and the interaction is not recognized! Drop-outs tend to have lower BMD initially.

19 / 60 20 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Objectives of calcium study Case: Missing data in calcium study Likely causes and effects of drop-out: Two possibilities for defining the target treatment effect: ◮ We expect that the positive effect of calcium ceases when the girl drops out of the trial (NMAR for target 1). 1. Difference in de facto mean BMD at end of study for everyone. ◮ Family moves away or too busy to participate (MCAR for 2. Difference in mean BMD which would have been found had target 2 unless related to unmeasured confounders). everyone completed their assigned treatment. ◮ Parents learn at the visit that BMD is low and decides to Which is closest to the effect of an intervention in the population withdraw because they think the girl is on placebo but needs is not that obvious, since reasons for discontinuing treatment in treatment (MAR for target 2). real life may be different from reasons for dropping out of the ◮ Families with an unhealthy lifestyles are more likely to study. Presumably everyone tolerates both calcium and placebo so withdraw (MAR or NMAR for target 2, depending on whether counterfactual outcomes are not unreasonable. The individual the lifestyle factors are included as covariates in the model). wants to know what can I expect happens to me if I take this treatment not what happens if I don’t. Note: What type of missing data we are dealing with has to be argued - it cannot be tested or otherwise assessed from the data . 21 / 60 22 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

NMAR: Not missing at random Outline

What to worry about when you have missing data No statistical method can make up for data being NMAR. Missing data types It has been suggested to perform sensitivity analyses over a of plausible models for the missingness / unobserved data. Simple methods for handling missing data

However: Advanced methods for handling missing data ◮ All such models rely on unverifiable assumptions that can never be checked with the data. Missing data in population average models (binary data) ◮ We have very limited possibilities to perform such analyses with statistical software. Death and other intercurrent events in longitudinal studies

Evaluation

23 / 60 24 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Naive methods for handling missing data Complete case analysis

Make an analysis including only those individuals who are Unwarranted approaches that are often substantially biased : observed at all available time points. ◮ Complete case analysis ◮ Default choice for oldfashioned software (e.g. MANOVA). ◮ LOCF (or LVCF): Last observation (value) carried forward ◮ Valid under MCAR-assumption

◮ Mean value . Consequences: ◮ Predicted value imputation. ◮ Likely biased if there are specific reasons for the missingness. ◮ Inefficient (reduced power) because partial information from Beware: These methods are still popular among health science non-completers is lost. researchers because they are so easy to use! Use with caution

25 / 60 26 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Per protocol analysis Last observation carried forward (LOCF)

Target: If an individual has no observed value at time t, replace the ◮ Estimate treatment effect in those subjects who are able to missing value by the previously observed value. complete (tolerate) all of the treatments. ◮ For drop-outs, all subsequent values will be identical.

Suggested analysis: Consequences: ◮ Complete case (possibly with additional exclusion of violators). ◮ The estimated time effect is most likely biased.

Limitations: ◮ The natural variation is obscurred, so standard errors are most ◮ Subjects may drop from e.g. active treatment and placebo for likely biased. different reasons (lack of effect vs adverse events). ◮ How do we identifiy those who tolerate all treatments? Run-in phase? Cross-over trial? Definitely not recommended

27 / 60 28 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

LOCF in clinical trials Predicted value imputation Replace the missing values with more or less qualified LOCF has been widely applied in clinical trials in the past. guesses of what they might have been, e.g. ◮ Mean value imputation: The LOCF is the easiest imputation approach for missing data to Replace missing value with average over observed data. be understood by the non-. However, the LOCF ◮ Model prediction imputation: approach has been the target for criticisms from the statisticians Replace missing values with predicted values from regression for its lack of a sound statistical foundation and for its in models including previous observations and other covariates. either direction (i.e., it is not necessarily conservative). After the National Academies published its draft report "The prevention and treatment of missing data in clinical trials”, using LOCF approach Consequences: seemed to be out-dated and markedly out of step with modern ◮ Estimates may be biased if the prediction model is not correct. statistical thinking. ◮ Likely to underestimate SEs because imputed values are treated as if they were the actual observations and because Reference: http://onbiostatistics.blogspot.dk/2012/05/is-last-observation- predictions are less variable than genuine observations. carried-forward.html Definitely not recommended 29 / 60 30 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Calcium: Predicted values for calcium drop-out Outline

What to worry about when you have missing data

Missing data types

Simple methods for handling missing data

Advanced methods for handling missing data

Missing data in population average models (binary data)

Death and other intercurrent events in longitudinal studies

Evaluation Predicted group mean / individual value based on the cLMM.

31 / 60 32 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Advanced methods for handling missing data Residuals for LMMs

Up-to-date methods : Raw residuals: Observed - Predicted. ◮ ◮ Likelihood inference (default in LMMs, e.g. proc mixed). Note that may change with time . ◮ Multiple imputations. Pearson residuals: (Obs - Pred) / SD(Obs).ˆ ◮ Inverse probablility weighting (IPW). ◮ Same variance, but residuals on same subject are correlated .

Usually the best available options for handling missing data Studentized residuals: (Obs - Pred) / SD(Obs-Pred).ˆ ◮ Valid under MAR (but does MAR hold?) ◮ Same as Pearson residuals in large samples; takes estimation ◮ Extensions to some particular NMAR scenarios. uncertainty in predictions into account when estimating SD. Scaled residuals: vciry-option in proc mixed. BUT: ◮ Scales residuals by the square root of the inverse V- matrix . ◮ Results aren’t as robust to modelmisspecification as with ◮ Independent and standard normal if the model is correct; complete data. E.g. the normal distribution matters . also when there are missing data as long as MAR holds.

33 / 60 34 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Goodness of fit, scaled residuals Likelihood inference

Perform the usual LMM analysis. ◮ In both SAS and R likelihood inference is default. ◮ All observations must be included, not just complete cases.

Properties: ◮ Valid under MAR if the model is correct . ◮ Efficient - makes optimal use of the available observations. ◮ Also applies to generalized linear mixed models for non-normal outcomes ( lecture 5 ).

◮ Not applicable to population average models (lecture 5 ).

Recommended. Model: categorical-time effect and unstructured .

35 / 60 36 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

The MAR assumption - again Multiple imputations

The MAR assumption can never be verified from the data. Similar to predicted value imputation only random error terms are added to predictions (and model parameters) . What you need to argue : 1. This is repeated M times and analysis is performed on each ◮ MAR means that responses from subjects who remain are imputed dataset. representative of everyone in the study population who has Recommendation: M = 5 , 100, 1000, depending on who you ask! similar characteristics (covariates) and a similar response until 2. Finally estimates are averaged and SEs are computed the time of drop out . according to Rubin’s rule . ◮ WHY do subjects drop out? Properties ◮ But even better: Valid if the imputation model is correct . ◮ ◮ Envision different NMAR-scenarios. Applicable to MAR and some NMAR scenarios . ◮ ◮ Make sensitivity analyses to check how results are affected But limited modeling possibilities in software . . . . as far as statistical software permits :( Recommended.

37 / 60 38 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

The imputation model I The imputation model II Quantitative missing data after drop out are imputed sequentially Multiple imputations are more difficult to perform with:

◮ conditionally on observed data before drop out, 1. Data that is not quantitative or of mixed type, ◮ . . . and previous imputed values, 2. Intermittent missing values or missing covariates. ◮ . . . and other predictors (covariates, auxiliary variables). since this involves

from standard models: 1. Using regression models of other/mixed type, e.g. logistic. 2. Repeating sequential imputations in a cyclic manner while Yi,j +1 = β˜j, 0 + β˜j, 1Yi, 1 + . . . + β˜j,j Yi,j + ( other ) + εi,j conditioning on all other input variables (other outcomes etc.).

If the outcome follows a multivariate normal distribution (given the Two different algorithms for this are available: predictors), then the imputation model is consistent with this. ◮ The substantive model compatible fully conditional Technical note: Model parameters have to be estimated as an integral step in specification algorithm (R-package SMCFCS). Recommended . the imputation procedure. This is technical and usually done using a so-called ◮ Multiple imputations by chained equations (R-package: mice, MCMC-algorithm (estimation and imputation is alternated until convergence). SAS: proc mi). Although not fully consistent with any multivariate To further reflect estimation uncertainty model parameters for each separate model, the algorithm seems to be working well in practice. imputation39 / 60 are sampled with error. This is called proper imputations . 40 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Calcium study: sensitivity analyses Calcium study: Multiple imputation plan.

When a girl drops out of the calcium group . . . ◮ A. Standard multiple imputation are carried out for each of the two groups, i.e. by grp in SAS with proc mi . ◮ A. The positive effect of calcium persist after drop out; the girl continues to gain in BMD like the remaining girls in the ◮ B. Drop outs are pooled with the placebo group and multiple calcium group (like in the counter factual world where she imputations are performed for this group alone. Resulting data stayed on treatment). from each imputation is joined with the calcium completers before analysis. ◮ B. The girl will keep the gain in BMD she got while on calcium, but any further gain she experiences after drop out ◮ C. Same as B except that all follow-up responses are deleted will be like in the placebo group. from the calcium drop outs before pooling their data with the placebo group. ◮ C. It is like she never had any calcium at all; at end of study her BMD will be like if she had been in the placebo group all ◮ Imputed datasets are analyzed with the cLMM (proc mixed) along. and resulting output is synthisized with proc mianalyze. Optimistic - realistic - pessimistic in terms of treatment effect. A. is mplemented in calcium2_demo.sas

41 / 60 42 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Calcium: Multiply imputed data (scenario A) Calcium study: 3 scenarios

43 / 60 44 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Calcium: Results of sensitivity analyses Agreement between LMM and multiple imputations

⋆ Method Estimate Std.Error P-value results and multiple imputations agree if: complete case cLMM 18.99 6.53 0.0046 1. the MAR-assumption holds (as in scenario A). cLMM same covariance 18.95 6.26 0.0032 2. the mixed model used for the analysis is the same as the cLMM different 18.79 6.30 0.0036 imputation model; i.e. an unrestricted model for the mean and covariance in each group seperately. Multiple imputations A 18.72 6.37 0.0033 3. sample size is large. Multiple imputations B 16.74 6.27 0.0076

Multiple imputations C 16.04 6.44 0.0128 In moderate and small samples multiple imputations tend to yield ⋆ Difference in mean BMD-gain at final follow-up (calcium vs placebo). conservative standard errors.

Conclusion: Effect of calcium is an overalll robust finding unless . . . but note that the p-values from proc mianalyze are based on some of the drop-outs are substantially different from other girls the normal approximation (no correction on degrees of freedom). (any unmeasured confounders. . . ?). 45 / 60 46 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Outline Case study: Amenorrhea (from lecture 5) 1151 women were randomized to a contracepting drug in either What to worry about when you have missing data ◮ low dose of 100 mg ( trt=0) / high dose of 150 mg ( trt=1)

Missing data types Are frequencies of amenorrhea biased due to missing data?

Analysis Variable : amenorrhea

Simple methods for handling missing data N N dose time Obs N Miss Mean Variance ------Advanced methods for handling missing data 0 1 576 576 0 0.1857639 0.1515187 2 576 477 99 0.2620545 0.1937882 Missing data in population average models (binary data) 3 576 409 167 0.3887531 0.2382065 4 576 361 215 0.5013850 0.2506925 Death and other intercurrent events in longitudinal studies 1 1 575 575 0 0.2052174 0.1633874 2 575 476 99 0.3361345 0.2236179 Evaluation 3 575 389 186 0.4935733 0.2506029 4 575 353 222 0.5354108 0.2494527 ------

47 / 60 48 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Missing data in PA models Inverse probability weighting (IPW)

Missing data should not be ignored as the GEE-estimates may ◮ Assign more weight to those who should be more inclined to become biased . drop out but actually stays in the trial (weighted GEE). E.g. we have a problem assessing the prevalences if ◮ If women with current disease are twice as likely to drop out ◮ Replicates are correlated; some women are more prone to than those without, those who stay have to count for two. experience amenorrhea than others. Properties: ◮ Women with amenorrhea at the previous occation are more likely to drop out. ◮ Valid under MAR-assumption if both the model for the ◮ Then women with amenorrhea would be underrepresented at missingness and the model for the observations are correct . later occations. ◮ In case of monotone missingness, it suffices that one of the ◮ Note that we are interested in the counterfactual prevalence two models is correct (estimates are doubly robust). of amenorrhea had everyone completed the study. Presumably ◮ No good with small dataset or high prevalence of missing data . the side effect ceases when a women comes off the drug, but that is not what we are interested in. Recommended when feasible .

49 / 60 50 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Modeling inverse probability weights Estimates with inverse probability weighting Need to model the response indicators at each occation. IPW compared to complete case analysis . ◮ Usually including previous outcomes, etc. ◮ Cumulate predicted probabilities of response over occations Estimate GEE with IPWs Complete cases ------â · · â πˆij = P (Ri1 = 1) . . . P (Rij = 1) risk time 1 (low) 0.19 (0.16;0.22) 0.18 (0.14;0.22) risk time 2 (low) 0.27 (0.23;0.31) 0.25 (0.21;0.30) ◮ −1 Input their inverses, wij =π ˆ ij as weights in GEE risk time 3 (low) 0.40 (0.35;0.44) 0.37 (0.32;0.42) (FLW chapter 19 describes how to do this in SAS). risk time 4 (low) 0.52 (0.46;0.57) 0.50 (0.45;0.55)

Drop out according to treatment and previous outcome . risk time 1 (high) 0.21 (0.17;0.24) 0.16 (0.13;0.20) risk time 2 (high) 0.34 (0.30;0.39) 0.30 (0.25;0.35) Occation2 dose amenorrhea1 N freq of drop out risk time 3 (high) 0.52 (0.47;0.57) 0.48 (0.43;0.53) ------risk time 4 (high) 0.57 (0.52;0.62) 0.54 (0.48;0.59) 0 0 469 0.16 1 107 0.21 1 0 457 0.15 Complete case analysis underestimates prevalences of amenorrhea 1 118 0.26 even at first occation where no one has dropped out yet! ------51 / 60 52 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Outline Intercurrent events

Non-existent data due to death is not the same as missing data. What to worry about when you have missing data In other circumstances collected data may not contain the Missing data types information that was intended, e.g. ◮ data from non-adherent patients does not reflect optimal / Simple methods for handling missing data true treatment efficacy. ◮ data after surgery does not reflect natural disease progression. Advanced methods for handling missing data An intercurrent event is an event that occurs after treatment Missing data in population average models (binary data) initiation and either preclude observation of the variable or affects its interpretation ⋆. Death and other intercurrent events in longitudinal studies ◮ How do we analyze data with intercurrent events - ???

Evaluation ⋆ ICH E9 (R1) addendum on estimands and sensitivity analyses in clinical trials to the guideline on statistical principles for clinical trials. EMA, August 2017 .

53 / 60 54 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Dead is not missing Analyzing randomized studies with death Potential difference in survival: ◮ Death must be considered a poor outcome. ◮ Compare summary that identifies an early death as a poor outcome. ◮ E.g. survival time or AUC until death/end of follow-up.

No difference in survival: ◮ Evaluate treatment effect in survivors only. ◮ Compare that identifies good/poor Scenario 1: Someone dies, no one drops out. outcome regardless of survival. ◮ Mean of observed data is an unbiased estimate of the population ◮ E.g. Average outcome while alive. mean (since the population consists of the survivors). Scenario 2: Someone drops out, no one dies. Missing data can be handled by making multiple imputations ◮ Estimated mean from linear mixed model is unbiased , in strata defined by time of death. 55 / 60 (assuming MAR and correct model specification). 56 / 60 university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Case: Tumor growth in mice Case:

Experimental treatment (n=8) compared to control (n=7). Outcome: time to sacrifice.

ATT: Sacrifice mice when tumor volume exceeds 1000 mm 3. ◮ Missing data is obviously MAR - why not impute? Alternatively: Time to doubling or quadrupling.

57 / 60 58 / 60

university of copenhagen department of biostatistics university of copenhagen department of biostatistics

Outline Course evaluation

What to worry about when you have missing data

Missing data types

Simple methods for handling missing data Your feedback is much appreciated! Advanced methods for handling missing data

Missing data in population average models (binary data)

Death and other intercurrent events in longitudinal studies

Evaluation

59 / 60 60 / 60