Analytic Considerations for Repeated Measures of Egfr in Cohort Studies of CKD

Feature

Analytic Considerations for Repeated Measures of eGFR in Cohort Studies of CKD

Haochang Shou,*† Jesse Y. Hsu,*† Dawei Xie,*† Wei Yang,*† Jason Roy,*† Amanda H. Anderson,*† J. Richard Landis,*† Harold I. Feldman,*† Afshin Parsa,‡§ and Christopher Jepson*† on behalf of the Chronic Renal Insufﬁciency Cohort (CRIC) Study Investigators

Abstract Repeated measures of various biomarkersprovide opportunities forus toenhanceunderstanding ofmanyimportant *Department of clinical aspects of CKD, including patterns of disease progression, rates of kidney function decline under different Biostatistics, risk factors, and the degree of heterogeneity in disease manifestations across patients. However, because of unique Epidemiology and features, such as correlations across visits and time dependency, these data must be appropriately handled using Informatics and †Center for Clinical longitudinal data analysis methods. We provide a general overview of the characteristics of data collected in cohort Epidemiology and studies and compare appropriate statistical methods for the analysis of longitudinal exposures and outcomes. We Biostatistics, Perelman use examples from the Chronic Renal Insufficiency Cohort Study to illustrate these methods. More specifically, we School of Medicine, model longitudinal kidney outcomes over annual clinical visits and assess the association with both baseline and University of longitudinal risk factors. Pennsylvania, Philadelphia, Clin J Am Soc Nephrol 12: 1357–1365, 2017. doi: https://doi.org/10.2215/CJN.11311116 Pennsylvania; ‡Department of Medicine, Division of Nephrology, Introduction In this paper, we will focus primarily on the appro- University of The term repeated measures refers to data observed priate statistical methods for analyzing longitudinal data Maryland School of repeatedly within the same subject, and they are being as outcomes. In particular, we will discuss exten- Medicine, Baltimore, Maryland; and increasingly collected in many research studies. Aside sively regression methods that can estimate the rate of § from being used to evaluate the reproducibility and Department of change of the longitudinal data in association with Medicine, Baltimore variability of a novel biomarker (1,2), repeated mea- certain risk factors (10,11). The terms repeated mea- Veterans Affairs sures are often generated in the context of longitudinal sures and longitudinal data are used interchangeably. Medical Center, studies, in which one or more biomarkers are ob- Baltimore, Maryland served over time. For chronic diseases, such as CKD, patterns of biomarker trajectories are crucial for un- Motivating Example Correspondence: derstanding of disease prognosis. The Chronic Renal Insufficiency Cohort (CRIC) Study Dr. Haochang Shou, Department of In many studies, the repetitions are predetermined by is a prospective, longitudinal study of patients with CKD, Biostatistics, the study protocol, whereby measures are administered in which repeated measures of serum creatinine and Epidemiology and prospectively at specific intervals during scheduled cystatin C, along with several other laboratory measures, Informatics, Perelman clinical visits or telephone interviews (3,4). In other were collected from each participant during annual School of Medicine, University of scenarios, the repeated data (e.g.,measuresofvital clinical visits (3,7,12). eGFR, calculated on the basis of Pennsylvania, 219 signs, such as BP, heart rate, and respiratory rate) the CRIC Study eGFR equation that incorporates both Blockley Hall, 423 become available at variable time points when certain serum creatinine and cystatin C (13), is an important Guardian Drive, events (e.g., hospitalization) occur. Repeated measures measure of kidney function (12). Describing the rate Philadelphia, PA could also be obtained retrospectively as natural history of change and patterns of eGFR decline as well as 19104.Email: hshou@ mail.med.upenn.edu data through available databases, such as Medicare (5). identifying risk factors that affect CKD progression are Depending on the scientific questions, the longitu- of particular interest in CKD research (10,14,15). dinal measures may serve as the outcomes of interest, Our motivating example involves the effect of func- the exposures, or a combination of both. Examples of tional kidney risk variants in the gene coding for APOL1 these scenarios in kidney disease research include ontherateofeGFRdeclineamongtheCRICStudy comparisons of the burden of coronary artery calcifi- participants. Two haplotypes (G1 and G2) in APOL1 cation for patients at different stages of CKD and have been positively selected and are common in ESRD, treating the longitudinal coronary artery calci- populations of recent African continental descent, but fication measures as the outcome of interest (6); they are very rare or absent in most other populations, evaluations of the associations of longitudinal mea- where exposure to Trypanosomes was not common. sures of GFR with subsequent adverse events, such as These APOL1 variants associated with kidney diseases are ESRDanddeath,asoutcomes(7,8);andaninvesti- believed to account for much of the nonsocioeconomic- gation of the causal relationship between BP and based related disparity in rates of CKD progression kidney function, in which both measures were up- between patients with African ancestry and white dated over time (9). patients.

DNA samples from the 1411 African ancestry participants fasterdeclinecomparedwithothers.Second,suchasinmost enrolled in the CRIC Study between June of 2003 and August of cohort studies, subjects have varying numbers of visits. Third, 2008 were genotyped for APOL1 risk variants (16). Given the the baseline eGFR values differ across subjects. near absence of these APOL1 risk variants in whites, the In the era of big data, many multivariable measures, such exposure variable was deﬁned in conjunction with race into as imaging scans, proteomics assessments, and electronic three categories: APOL1 high-risk genotype (African ancestry health records (6,19,20), are collected at multiple visits for participants with two copies of the risk variants), APOL1 low- large cohorts of participants. It might no longer be feasible risk genotype (African ancestry participants with zero or one to include all of the measures in one data frame. Utilization of copy of these variants), and white participants (reference group). high-performance computing and central databases, such as The investigators were interested in assessing whether the National Institute of Diabetes and Digestive and Kidney rates of eGFR decline differ among the three exposure Diseases Repository (21,22), that crosslink various measures groups. The longitudinal outcome in this example was the to a unique subject-visit identity is crucial to handle complex annual eGFR measures for each participant for up to 7 years and heterogeneous data. In addition, advanced statistical after enrollment. Other covariates included demographics tools, including dynamic visualization interface and hierar- (e.g.,age,sex,andclinicalsite),socioeconomicvariables chical clustering visualization using graph structures (23,24), (e.g., income and education level), and traditional clinical need to be leveraged to CKD research involving big data risk factors (e.g., systolic BP and body mass index) and were (25–28). mostly observed at baseline (16).

Time Dependency and Correlated Observations Data Preparation and Visualization Longitudinal data have unique and crucial characteris- For longitudinal studies of moderate size, the dataset is often tics. First, they are typically accompanied by a time variable prepared in either of the two ways: wide format (one row of that indicates when each measurement occurred, and they data per participant) or long format (multiple rows of data per deﬁne a natural ordering of the repeated measures within participant; one per visit) (17) (details are in Supplemental each subject. Second, the repeated measures within the Appendix 1). The long format is generally more preferred for same subject are potentially correlated. For example, within advanced statistical modeling, because it can handle subjects each subject, the eGFR values of subsequent visits might with different numbers of clinical visits or irregular time points depend on those at earlier visits. Such intrinsic clusters of measurement; it is also easier for dynamically updating the deﬁned by subjects result in dependency (correlation) dataset with future follow-up visits. among repeated observations, which violates the assump- Exploratory analysis using graphs can help researchers to tion of independent observations on which many simple frame the hypothesis and select appropriate statistical models. analytic methods are based. Hence, using traditional linear Some commonly used visualization tools for longitudinal data regression and ignoring data correlations might lead to include spaghetti plot, heat map, and lasagna plot (18) (Figures inaccurate estimates and erroneous inferences about the 1 and 2). These plots show several features of the eGFR associations of risk factors with kidney function decline trajectories. First, patients with APOL1 high risk seem to have a (29).

Figure 1. | Spaghetti plot of eGFR over time for 15 random Chronic Renal Insufficiency Cohort (CRIC) Study participants. The figure shows an example of the spaghetti plot of repeated eGFR values for 15 randomly selected CRIC Study participants, five from each of the three APOL1 risk categories. The spaghetti plot connects the longitudinal eGFR values within a single subject by lines over time but might result in overplotting with too many subjects on one plot. Clin J Am Soc Nephrol 12: 1357–1365, August, 2017 Repeated Measures in CKD, Shou et al. 1359

Figure 2. | Heat map (left panel) and lasagna plot (right panel) plot on the basis of the same 15 subjects. Each row of the heat map represents eGFR values from one subject, and the color indicates the level of eGFR values. The lasagna plot is a heat map sorted by color gradients to reﬂect the overall eGFR distributions in the population at each visit.

Special statistical methodologies for repeated mea- study population and estimates the marginal associations sures allow observations to be correlated and can quantify between the repeated outcome measures and the risk factors. the magnitude of this correlation. In addition, some of these A GEE model consists of two parts: the mean response model approaches are designed to predict the individual trajectory of and the error term. In our motivating example, we can ﬁta the outcome measures over time and can handle datasets with linear GEE model as expressed in model 1 below. For missing values. Such methods often make assumptions about simplicity, no additional covariates are included in this the structure of correlations between the repeated measures. model,exceptforthemajorriskfactor:APOL1 risk group (0, Typical choices of correlation structures include indepen- white; 1, African ancestry APOL1 low risk; and 2, African dence, exchangeable, autoregressive, m dependent, and un- ancestry APOL1 high risk): structured (30,31) (Supplemental Appendix 2). eGFR ; APOL1 1 year 1 year 3 APOL1 1 error term: (1)

Statistical Approaches for Repeated Measures as the ThemeanresponsemodeleGFR~ APOL1 1 year 1 Outcome year 3 APOL1 takes the repeated measures (e.g.,eGFR In our motivating example, the goal of the analysis was to at jth visit for subject i) as the outcome and describes the investigate the rate of kidney function decline and its associ- overall mean relationship between repeated eGFR values and ation with related risk factors. Intuitively, a two-stage ap- the exposure variables. We present a rigorous formulation proach has been used (10,32,33) in the literature. Take eGFR as ’ and detailed interpretation of model 1 in Supplemental an example. First, an individual s eGFR slope is separately es- fi timated for each subject by regressing the subject’s repeated Appendix 3, but we point out that the coef cient estimated eGFR values on the time variable. Second, the eGFR slopes in front of the time variable (year) represents the average from all of the subjects are treated as the outcome variable and eGFR slope for the reference group (whites) and that the fi fitted into a linear regression. Such a method is limited, in that coef cient for the interaction of year with APOL1 estimates the the estimates are highly sensitive to random variations (noise), difference in eGFR slopes between the other two APOL1 risk especially for subjects with short follow-up and small numbers groups compared with the reference. In addition, the associ- of observations. It is also not able to evaluate the cross- ations between baseline eGFR and covariates are represented sectional associations between baseline eGFR and risk factors via coefficients in front of the nontime-varying variables in a or adaptable to time-varying covariates (34). similar fashion as a cross-sectional linear regression. To simultaneously model the effects of risk factors on both The error term accounts for the within-subject correlation. the baseline observations and their rate of change and to appro- GEE initiates the model estimation by specifying a particular priately account for the correlations among the repeated correlation structure among the repeated observations re- measures, two types of flexible regression methods are recom- ferred to as the working correlation, which is not necessarily mended for the analysis of longitudinal data: generalized esti- accurate for the real data. The working correlation is typically mating equations (GEEs) and mixed effects modeling (17,29,34). chosen from one of the aforementioned common structures: We introduce these methods in the following sections. independence, exchangeable, autoregressive, m dependent, and unstructured (Supplemental Appendix 2). For example, Population-Average Model exchangeable structure assumes that the correlation between A population-average model, such as the GEE approach outcomes is equal, regardless of how far apart they are in time. (17,35), captures the average trajectories across the overall Autoregressive is often used in longitudinal analysis and refers 1360 Clinical Journal of the American Society of Nephrology

to when the correlation decreases as the time difference In our example, the subject-specific random intercept between observations increases. The unstructured correlation characterizes the difference between an individual’sbase- structure is the most flexible, with no prespecified patterns. line eGFR and the population average, and a random slope GEE is robust in that an erroneously specified working in front of a time variable describes the deviation of correlation structure will have little effect on the risk asso- individual slope from the population average. Thus, a ciation estimation as long as the mean response model is simple mixed effects model that is analogous to model 1 is correct. However, choosing a working correlation that closely as follows: approximates the truth is still desirable, because it results in smaller standard errors in the final estimates (30,31,36). eGFR ; APOL1 1 year 1 year 3 APOL1 With the data from the motivating example, we fit 1 random effects 1 error term: (2) model 1 assuming an exchangeable working correlation fi structure and present the estimated coef cients in Table 1. The fixed effects coefficients share the same interpretations The results show that eGFR, on average, decreases 0.50 ml/min as in the GEE model 1. It is the subject-specific random 2 per 1.73 m per year for whites. The eGFR decline among the effects that quantify heterogeneity across individuals. 2 APOL1 high-risk group is 0.94 ml/min per 1.73 m per year Although they both estimate individual eGFR slopes, the , faster than among whites (P 0.001). The APOL1 low-risk mixed effects model is different from the aforementioned group also has a faster decline than in whites, but their two-stage approach in that it manages to use information 2 difference is milder (0.38 ml/min per 1.73 m more per year). from all of the subjects by assuming a common distribution (typically, a normal distribution with mean of zero) for the Subject-Specific Model random effects, and hence, it avoids the problem of In addition to the overall mean effects of APOL1 on the estimating too many parameters with too few observations. eGFR trajectory, different subjects could start with various The random effects also take care of the correlation among eGFR values at baseline and also progress differently. In a repeated observations within subject. Additional working GEE model, the subject heterogeneity is absorbed into the correlation structures can be further imposed onto the error error term, and its magnitude is thus not quantified. Mixed term, making it adaptable to complex correlation struc- effects modeling, which has been more commonly used tures. A linear model with only a random intercept and than GEE in the nephrology literature (11), estimates both independent error term would induce an exchangeable the individual variations in baseline outcome variables and correlation structure. their rates of change that deviate away from the popula- Tables 2–3 show the estimates for fixed effects coeffi- tion-average trajectory. This is achieved by adding random cients and the estimated subject-specific random effects in effects (random intercept and/or random slope) to the model 2. On average, eGFR decreases 0.74 ml/min per population-average model (referred to as fixed effects), as 1.73 m2 per year for white participants. The eGFR decline illustrated in Figure 3, by two hypothetical subjects. among the APOL1 high-risk group is significantly faster by

Table 1. Interpretation of coefﬁcients from a generalized estimating equation model assessing associations of APOL1 genotype with eGFR measured repeated over time

Coefficient Coefficient Variablea Coefficient Interpretation P Value Estimate SEM

APOL1 high risk Difference in average baseline eGFR (milliliters per 23.36 1.13 0.003 minute per 1.73 m2) among African ancestry participants with APOL1 high-risk alleles compare with whites APOL1 low risk Difference in average baseline eGFR (milliliters per 23.87 0.63 ,0.001 minute per 1.73 m2) among African ancestry participants with APOL1 low-risk alleles compare with whites Years Average eGFR slope (milliliters per minute per 1.73 m2 20.50 0.06 ,0.001 per year) among whites Years 3 APOL1 Difference in average eGFR slope (milliliters per minute 20.94 0.21 ,0.001 high risk per 1.73 m2 per year) among African ancestry participants with APOL1 high-risk alleles and whites Years 3 APOL1 Difference in average eGFR slope (milliliters per minute 20.38 0.11 ,0.001 low risk per 1.73 m2 per year) among African ancestry participants with APOL1 low-risk alleles and whites

Results are from a generalized estimating equation model fit for the Chronic Renal Insufficiency Cohort Study example in model 1 under exchangeable correlation structure. aModel 1: eGFR~APOL1 1 year 1 year 3 APOL1 1 error term: APOL1 is the exposure variable of the genotype in conjunction with race with three categories (0, white; 1, APOL1 low risk; and 2, APOL1 high risk), with whites as the reference group. The reported coefficient SEMs are the robust estimates combining empirical data correlation and the assumed working correlation structure. Clin J Am Soc Nephrol 12: 1357–1365, August, 2017 Repeated Measures in CKD, Shou et al. 1361

Figure 3. | Schematic illustrations of linear mixed effects models with random intercepts (left panel) and both random intercepts and random slopes (right panel). The blue and red dots represent observed eGFR values for the two hypothetical subjects i and i9. The blue and red curves represent the mixed effects model ﬁt on the basis of the data. The black lines represent the population trend estimated by generalized estimating equation (GEE). With the random intercept model, the two subjects have different baseline eGFR values but the same rate of decline. In the random intercept model, their rates of decline also differ. The deviations of intercepts and slopes are quantiﬁed using random effects parameters.

1.50 ml/min per 1.73 m2 per year than among whites covariate, such as in the work by Parsa et al. (16), when the (P,0.001). sample size per site is sufficiently large or (2) adding a Another advantage of the mixed effects model is that it is site-specific random effect in the mixed effects model to flexible enough to account for multiple layers of clustering. quantify the intra- versus intersite variability. In particular, many cohort studies, including the CRIC Study, recruit participants from multiple clinical sites. In Model Diagnosis addition to the within-subject correlation among repeated Model selection is often conducted to choose a set of measures, data correlation could also occur across subjects variables that is most relevant to the outcome or select who come from the same geographic location or social the best working correlation structure. Unlike GEE, the community. It is often necessary to adjust for site effects correlation structure of the mixed effects model must be during the analysis by either (1) including site as a discrete specified in advance, and the results may be affected by

Table 2. Interpretation of the ﬁxed effects coefﬁcients from a linear mixed effects model assessing associations of APOL1 genotype with eGFR measured repeated over time

Coefficient Coefficient P Variablea Coefficient Interpretation Estimate SEM Value

APOL1 high Difference in average baseline eGFR (milliliters per 22.95 1.05 0.005 risk minute per 1.73 m2) among African ancestry participants with APOL1 high-risk alleles compare with whites APOL1 low risk Difference in average baseline eGFR (milliliters per 23.77 0.62 ,0.001 minute per 1.73 m2) among African ancestry participants with APOL1 low-risk alleles compare with whites Years Average eGFR slope (milliliters per minute per 1.73 m2 20.74 20.07 ,0.001 per year) among whites Years 3 APOL1 Difference in average eGFR slope (milliliters per 21.50 0.20 ,0.001 high risk minute per 1.73 m2 per year) among African ancestry participants with APOL1high-risk alleles and whites Years 3 APOL1 Difference in average eGFR slope (milliliters per 20.58 0.11 ,0.001 low risk minute per 1.73 m2 per year) among African ancestry participants with APOL1 low-risk alleles and whites

Results are the estimated fixed effects from a random slope model fit for the Chronic Renal Insufficiency Cohort Study example in model 2. aModel 2: eGFR~APOL1 1 year 1 year 3 APOL1 1 random effects 1 error term: APOL1 is the exposure variable of the genotype in conjunction with race with three categories (0, white; 1, APOL1 low risk; and 2, APOL1 high risk), with whites as the reference group. The random effects include both random intercept and random slope to account for subject-specific deviation from the population-average trajectory. 1362 Clinical Journal of the American Society of Nephrology

Table 3. Predicted individual deviation (random intercept and slope) in model 2 for ﬁve subjects in the white group

Subject APOL1 Risk Random Intercept, ml/min per 1.73 m2 Random Slope, ml/min per 1.73 m2 per year No. Category

1 216.35 20.04 White 2 1.85 0.81 White 3 10.22 1.27 White 4 22.31 22.56 White 55.58 23.38 White misspecification. For mixed effects models, the one that has transformation is commonly used if the original outcome data smaller Aikake Information Criterion (AIC) and Bayesian decline exponentially or have a skewed distribution. The lim- Information Criterion (BIC) generally fits the data better. A itation of this approach is that the estimated rate of kidney likelihood ratio test (37) could also be used to compare two function decline on the basis of the transformed data often models with and without a certain variable. For GEE, one lacks a straightforward clinical interpretation (11). can instead choose the best model that minimizes either the Alternatively, functional data analysis techniques are a set of quasilikelihood under the independence model criterion flexible methods to characterize the underlying smooth tra- (QIC) (Table 4) (38), or the QICu defined as QIC+2p that jectoriesonthebasisoftheobserved longitudinal measure- penalizes the model when too many variables are included. ments. For example, we can model the eGFR values as a Comparing the empirical correlation on the basis of the polynomial (e.g., quadratic or cubic) function of time in the observed data with the model-based estimates assuming regression model. In our motivating example, models 1 and 2 that the working correlation is true or conducting sensitivity can also include new variables year2 or year3 in addition to the analyses to examine how the choice of working correlation linear term year, and the statistical significance of the higher- structures affects the analytic results are both crucial ways order polynomial terms can then be tested to determine for appropriate model selection in GEE. whether a nonlinear eGFR trajectory is truly present. More broadly, smoothing splines (e.g., B spline) are used to accom- Non-Normal Outcomes modate the curvilinear trends, such as piecewise linear or For longitudinal outcomes that are not normally distrib- polynomial (44,45). The two-slope model used for log serum uted, such as hospitalizations (discrete counts) and occur- creatinine in the Chronic Kidney Disease Epidemiology rence of AKI (binary), GEE and mixed effects models can Collaboration equation is a special case of the piecewise- both handle such data, analogous to the way in which the linear spline with one change point (knot) where the eGFR linear regression is extended to generalized linear models slope is believed to alter (46,47). Principal component (e.g., logistic or Poisson regression [37]). The extension to analysis–type approaches (48), however, identify the pat- non-normal outcomes for mixed effects models is called terns of the trajectories that explain most of the variations generalized linear mixed models (29). across subjects in a data-driven fashion. Some efforts have been devoted to identify subgroups of patients who show Nonlinear Trends distinct patterns in the repeated measures using latent class Another assumption often made for kidney function analysis, such as group-based trajectory modeling (49–51). decline is that the rate of change remains constant over These methods are powerful, but they often require more time (that is, the disease progresses linearly). This assump- observations or longer follow-up to generate reliable model tion is convenient but is not always realistic, because eGFR estimates of the nonlinear trajectories and are computation- decline could accelerate or stabilize at different stages ally more expensive than the traditional models. of CKD (39,40). Several statistical methods have been de- Missing Data and Joint Modeling for Informative Dropout veloped to accommodate the potential for nonlinear trajec- Missing data are almost inevitable in a longitudinal tories. The first approach is to conduct a data transformation study for various reasons, such as participants’ dropout on the outcome variable (e.g.,log,squareroot[41–43]). Log (52), data mishandling, or quality screening (53,54). Both

Table 4. Comparison of different working correlation structures using quasilikelihood information criterion

Working Estimate, ml/min per 1.73 m2 SEM, ml/min per 1.73 m2 per year P Value QIC Correlations

Independence 20.17 0.32 0.60 84,143.50 Exchangeable 20.94 0.21 ,0.001 84,137.98 Autoregressive 21.33 0.23 ,0.001 84,140.64

Coefﬁcients for year 3 APOL1,withAPOL1 52. Exchangeable correlation structure achieved the smallest QIC and hence, was more appropriate for the data. QIC, quasilikelihood information criterion. Clin J Am Soc Nephrol 12: 1357–1365, August, 2017 Repeated Measures in CKD, Shou et al. 1363

Table 5. Comparisons of the different aspects of the generalized estimating equation model and the mixed effects model for repeated measures

Regression Models GEE Mixed Effects

Model components Mean response model and error term Fixed and random effects and error term and parameters Non-normal outcome GEE with specified link functions GLMM with specified link functions Usage Association/predict population-average Association/predict both population- trajectory average and individual trajectories Goodness of fit metrics Quasilikelihood information criterion Aikake Information Criterion/Bayesian Information Criterion Correlation structure Prespecified working correlation (e.g., Correlation structure induced by both independence, exchangeable, random effects and error term; more autoregressive, m dependent, flexible in partitioning variability among unstructured) various hierarchies Missing assumptions Covariate-dependent MCAR; cannot MCAR and missing at random; cannot handle missing not at random or handle missing not at random or informative censoring informative censoring Pros and cons Robust for misspecification of correlation Suitable for data with high subject structures heterogeneity; higher computational cost

Repeated measures as outcome. GEE, generalized estimating equation; GLMM, generalized linear mixed model; MCAR, missing completely at random.

GEE and mixed effects model allow for imbalanced in the Supplemental Appendix 5. We are aware that there designs, in which the outcome data are not available at remain many limitations in the discussion. For example, topics, the same set of time points for all subjects. However, it is such as how to handle periods of dialysis or AKI episodes, that necessary to note that GEE makes more restricted assump- affect eGFR values were not covered, because it is difficult to tions in missing mechanism than mixed effects model (55) captureAKIduetotheinfrequencyofeGFRmeasuresinmost (Supplemental Appendix 4). Under missing at random, a longitudinal cohorts (71). Accordingly, our immediate goal is to weighted GEE has been developed (56) that corrects biased promote further discussions and emphasize a few key consid- estimates produced by the standard GEE. An alternative erations involved in carefully choosing analytic methods approach is to fill in the incomplete observations using primarily for outcomes on the basis of repeated measures as multiple imputations (57) followed by the standard GEE most appropriate to each specific study design, question, (58,59). Such methods are preferred, especially when the and measure. outcome data are not normally distributed (58–60). Note that imputation using last observation carried forward is Acknowledgments Funding for the CRIC Study was obtained under a cooperative not recommended for longitudinal data, because it is agreement from National Institute of Diabetes and Digestive and known to produce highly biased estimates (61,62). Kidney Diseases (U01DK060990, U01DK060984, U01DK061022, Informative censoring (52) or missing not at random (55) U01DK061021, U01DK061028, U01DK060980, U01DK060963, and often occurs in longitudinal studies when some patients drop U01DK060902). In addition, this work was supported in part by: the out of the study due to events that are related to CKD pro- Perelman School of Medicine at the University of Pennsylvania gression, such as initiation of dialysis, kidney transplantation, Clinical and Translational Science Award National Institutes of or death (63). In these scenarios, an individual with worse Health (NIH) / National Center for Advancing Translational Sci- kidney function condition (or a particularly low eGFR) is more ences (NCATS) UL1TR000003, Johns Hopkins University UL1TR- likely to drop out of the study and have their eGFR value un- 000424, University of Maryland GCRC M01 RR-16500, Clinical and observed. Appropriate methods that take the cause of miss- Translational Science Collaborative of Cleveland, UL1TR000439 ingness into account include conditional regression or pattern from the NCATS component of NIH and NIH roadmap for Medical mixtureanalysis(64,65)andselectionmodel(66)aswellasthe Research, Michigan Institute for Clinical and Health Research joint modeling of survival outcome (e.g.,ESRD)andlongitu- UL1TR000433, University of Illinois at Chicago CTSA UL1RR029879, dinal data, where a Cox regression model for time to dropout Tulane COBRE for Clinical and Translational Research in Car- and a mixed effects model for the longitudinal observations diometabolic Diseases P20 GM109036, Kaiser Permanente NIH/Na- are specified separately and linked together (67–70). tional Center for Research Resources UCSF-CTSI UL1 RR-024131. CRIC Study Investigators also include Lawrence J. Appel (Welch Summary Center for Prevention, Epidemiology and Clinical Research, Johns This paper focused mainly on appropriate statistical meth- Hopkins University, Baltimore, Maryland), Jiang He (Departments odologies for longitudinal data, where the repeated measures of Medicine and Epidemiology, Tulane University, New Orleans, are treated as the outcome variable, and they are summarized Louisiana), James P. Lash (Section of Nephrology, Department of in Table 5, with the comprehensive comparison between the Medicine, University of Illinois at Chicago, Chicago, IL), Akinlolu two regression methods, GEE and mixed effects model. The Ojo (Department of Medicine, University of Michigan, Ann Arbor, corresponding statistical softwares for these models are listed Michigan), and Raymond R. Townsend (Department of Medicine 1364 Clinical Journal of the American Society of Nephrology

and Center for Clinical Epidemiology and Biostatistics, University 15. Fischer MJ, Hsu JY, Lora CM, Ricardo AC, Anderson AH, Bazzano of Pennsylvania, Philadelphia, Pennsylvania). L, Cuevas MM, Hsu CY, Kusek JW, Renteria A, Ojo AO, Raj DS, Rosas SE, Pan Q, Yaffe K, Go AS, Lash JP; Chronic Renal In- Disclosures sufficiency Cohort (CRIC) Study Investigators: CKD progression and mortality among hispanics and non-hispanics. JAmSoc None. Nephrol 27: 3488–3497, 2016 16. Parsa A, Kao WH, Xie D, Astor BC, Li M, Hsu CY, Feldman HI, Parekh RS, Kusek JW, Greene TH, Fink JC, Anderson AH, Choi MJ, References Wright Jr. JT, Lash JP, Freedman BI, Ojo A, Winkler CA, Raj DS, 1. Rodriguez RA, Cronin V, Ramsay T, Zimmerman D, Ruzicka M, Kopp JB, He J, Jensvold NG, Tao K, Lipkowitz MS, Appel LJ; AASK Burns KD: Reproducibility of carotid-femoral pulse wave velocity Study Investigators; CRIC Study Investigators: APOL1 risk vari- in end-stage renal disease patients: Methodological consider- ants, race, and progression of chronic kidney disease. NEnglJ ations. Can J Kidney Health Dis 3: 20, 2016 Med 369: 2183–2196, 2013 2. Koufaki P, Naish PF, Mercer TH: Reproducibility of exercise tol- 17. Zeger SL, Liang KY: An overview of methods for the analysis of erance in patients with end-stage renal disease. Arch Phys Med longitudinal data. Stat Med 11: 1825–1839, 1992 Rehabil 82: 1421–1424, 2001 18. Swihart BJ, Caffo B, James BD, Strand M, Schwartz BS, Punjabi 3. Feldman HI, Appel LJ, Chertow GM, Cifelli D, Cizman B, NM: Lasagna plots: A saucy alternative to spaghetti plots. Epi- Daugirdas J, Fink JC, Franklin-Becker ED, Go AS, Hamm LL, He J, demiology 21: 621–625, 2010 HostetterT,HsuCY,JamersonK,JoffeM,KusekJW,LandisJR, 19. Mihai S, Codrici E, Popescu ID, Enciu AM, Rusu E, Zilisteanu D, LashJP,MillerER,Mohler3rd ER,MuntnerP,OjoAO,RahmanM, Albulescu R, Anton G, Tanase C: Proteomic biomarkers panel: Townsend RR, Wright JT; Chronic Renal Insufficiency Cohort New insights in chronic kidney disease. Dis Markers 2016: (CRIC) Study Investigators: The Chronic Renal Insufficiency 3185232, 2016 Cohort (CRIC) Study: Design and methods. JAmSocNephrol14 20. Sulemane S, Panoulas VF, Nihoyannopoulos P: Echocardio- [Suppl 2]: S148–S153, 2003 graphic assessment in patients with chronic kidney disease: 4. ARIC investigators: The Atherosclerosis Risk in Communities Current update. Echocardiography 34: 594–602, 2017 (ARIC) Study: Design and objectives. The ARIC investigators. 21. Cuticchia AJ, Cooley PC, Hall RD, Qin Y: NIDDK data repository: Am J Epidemiol 129: 687–702, 1989 A central collection of clinical trial data. BMC Med Inform Decis 5. Wanner C, Oliveira JP, Ortiz A, Mauer M, Germain DP, Linthorst Mak 6: 19, 2006 GE, Serra AL, Maro´di L, Mignani R, Cianciaruso B, Vujkovac B, 22. TurnerCF,Pan H,Silk GW,Ardini MA, Bakalov V,BryantS, CantorS, Lemay R, Beitner-Johnson D, Waldek S, Warnock DG: Prognostic Chang KY, DeLatte M, Eggers P, Ganapathi L, Lakshmikanthan S, indicators of renal disease progression in adults with Fabry dis- Levy J, Li S, Pratt J, Pugh N, Qin Y,Rasooly R, Ray H, Richardson JE, ease: Natural history data from the Fabry Registry. Clin J Am Soc Riley AF, Rogers SM, Scheper C, Tan S, White S, Cooley PC: The Nephrol 5: 2220–2228, 2010 NIDDK Central Repository at 8 years–ambition, revision, use and 6. Bansal N, Keane M, Delafontaine P,Dries D, Foster E, Gadegbeku impact. Database (Oxford) 2011: bar043, 2011 CA, Go AS, Hamm LL, Kusek JW, Ojo AO, Rahman M, Tao K, 23. Huang CW, Lu R, Iqbal U, Lin SH, Nguyen PA, Yang HC, Wang CF, Wright JT,Xie D, Hsu CY; CRIC Study Investigators: A longitudinal Li J, Ma KL, Li YC, Jian WS: A richly interactive exploratory study of left ventricular function and structure from CKD to ESRD: data analysis and visualization tool using electronic medical The CRIC study. Clin J Am Soc Nephrol 8: 355–362, 2013 records. BMC Med Inform Decis Mak 15: 92, 2015 7. Ku E, Xie D, Shlipak M, Hyre Anderson A, Chen J, Go AS, He J, 24. Riehmann P, Hanfler M, Froehlich B: Interactive Saankey diagrams, Horwitz EJ, Rahman M, Ricardo AC, Sondheimer JH, Townsend RR, information visualization. Presented at the IEEE Symposium on In- Hsu CY; CRIC Study Investigators: Change in measured GFR versus formation Visualization, Minneapolis, MN, October 23–25, 2005 eGFR and CKD outcomes. J Am Soc Nephrol 27: 2196–2204, 2016 25. Dinov ID, Heavner B, Tang M, Glusman G, Chard K, Darcy M, 8. Tseng CL, Lafrance JP, Lu SE, Soroka O, Miller DR, Maney M, Madduri R, Pa J, Spino C, Kesselman C, Foster I, Deutsch EW, Price Pogach LM: Variability in estimated glomerular filtration rate ND, Van Horn JD, Ames J, Clark K, Hood L, Hampstead BM, Dauer values is a risk factor in chronic kidney disease progression W, Toga AW: Predictive big data analytics: A study of Parkinson’s among patients with diabetes. BMC Nephrol 16: 34, 2015 disease using large, complex, heterogeneous, incongruent, multi- 9. Anderson AH, Yang W, Townsend RR, Pan Q, Chertow GM, Kusek sourceandincompleteobservations.PLoS One 11:e0157077,2016 JW, Charleston J, He J, Kallem R, Lash JP,Miller 3rd ER, Rahman M, 26. Zhang Y, Padman R: Data-driven clinical and cost pathways for Steigerwalt S, Weir M, Wright Jr. JT, Feldman HI; Chronic Renal chronic care delivery. Am J Manag Care 22: 816–820, 2016 Insufficiency Cohort Study Investigators: Time-updated systolic 27. Zhang Y, Padman R, Patel N: Paving the COWpath: Learning and blood pressure and the progression of chronic kidney disease: A visualizing clinical pathways from electronic health record data. cohort study. Ann Intern Med 162: 258–265, 2015 J Biomed Inform 58: 186–197, 2015 10. Rosansky SJ: Renal function trajectory is more important than 28. Holzinger A: Machine Learning for Health Informatics: State-of- chronic kidney disease stage for managing patients with chronic the-Art and Future Challenges, New York, Springer, 2016 kidney disease. Am J Nephrol 36: 1–10, 2012 29. Fitzmaurice G, Laird N, Ware J: Applied Longitudinal Analysis, 11. Boucquemont J, Heinze G, Jager KJ, Oberbauer R, Leffondre K: Hoboken, NJ, John Wiley & Sons, 2012 Regression methodsforinvestigating riskfactors of chronickidney 30. Diggle PJ, Heagerty P, Liang K-Y, Zeger SL: Analysis of Longitu- disease outcomes: The state of the art. BMC Nephrol 15: 45, 2014 dinal Data, New York, Oxford University Press, 2002 12. Yang W, Xie D, Anderson AH, Joffe MM, Greene T, Teal V,Hsu CY, 31. Lipsitz SR, Fitzmaurice GM, Orav EJ, Laird NM: Performance of Fink JC, He J, Lash JP, Ojo A, Rahman M, Nessel L, Kusek JW, generalized estimating equations in practical situations. Bio- Feldman HI; CRIC Study Investigators: Association of kidney metrics 50: 270–278, 1994 disease outcomes with risk factors for CKD: Findings from the 32. Barbour SJ, Er L, Djurdjev O, Karim M, Levin A: Differences in Chronic Renal Insufficiency Cohort (CRIC) study. Am J Kidney progression of CKD and mortality amongst caucasian, oriental Dis 63: 236–243, 2014 Asian and South Asian CKD patients. Nephrol Dial Transplant 13. Anderson AH, Yang W, Hsu CY, Joffe MM, Leonard MB, Xie D, 25: 3663–3672, 2010 Chen J, Greene T, Jaar BG, Kao P, Kusek JW, Landis JR, Lash JP, 33. Levin A, Djurdjev O, Beaulieu M, Er L: Variability and risk factors Townsend RR, Weir MR, Feldman HI, Investigators CS; CRIC for kidney disease progression and death following attainment of Study Investigators: Estimating GFR among participants in the stage 4 CKD in a referred cohort. Am J Kidney Dis 52: 661–671, Chronic Renal Insufficiency Cohort (CRIC) Study. Am J Kidney 2008 Dis 60: 250–261, 2012 34. Burton P, Gurrin L, Sly P: Extending the simple linear regression 14. Amdur RL, Feldman HI, Gupta J, Yang W, Kanetsky P, Shlipak M, model to account for correlated responses: An introduction to Rahman M, Lash JP, Townsend RR, Ojo A, Roy-Chaudhury A, Go generalized estimating equations and multi-level mixed model- AS, Joffe M, He J, Balakrishnan VS, Kimmel PL, Kusek JW, Raj DS; ling. Stat Med 17: 1261–1291, 1998 CRIC Study Investigators: Inflammation and progression of CKD: 35. Zeger SL, Liang KY: Longitudinal data analysis for discrete and The CRIC Study. Clin J Am Soc Nephrol 11: 1546–1556, 2016 continuous outcomes. Biometrics 42: 121–130, 1986 Clin J Am Soc Nephrol 12: 1357–1365, August, 2017 Repeated Measures in CKD, Shou et al. 1365

36. Carlin JB, Wolfe R, Brown CH, Gelman A: A case study on the 53. O’Kelly M, Ratitch B: Clinical Trials with Missing Data: A Guide for choice, interpretation and checking of multilevel models for Practitioners,Chichester,UnitedKingdom,JohnWiley&Sons,2014 longitudinal binary outcomes. Biostatistics 2: 397–416, 2001 54. Mallinckrodt C: Preventing and Treating Missing Data in Longi- 37. McCullagh P, Nelder JA: Generalized Linear Models, London, tudinal Clinical Trials: A Practical Guide, Cambridge, United Chapman & Hall, 1989 Kingdom, Cambridge University Press, 2013 38. Pan W: Akaike’s information criterion in generalized estimating 55. Rubin DB: Inference and missing data. Biometrika 63: 581–592, 1976 equations. Biometrics 57: 120–125, 2001 56. Preisser JS, Lohman KK, Rathouz PJ: Performance of weighted 39. Li L, Astor BC, Lewis J, Hu B, Appel LJ, Lipkowitz MS, Toto RD, estimating equations for longitudinal binary data with drop-outs Wang X, Wright Jr. JT, Greene TH: Longitudinal progression missing at random. Stat Med 21: 3035–3054, 2002 trajectory of GFR among patients with CKD. Am J Kidney Dis 59: 57. Rubin DB: Multiple Imputation for Nonresponse in Surveys, New 504–512, 2012 York, Wiley, 1987 40. Norris KC, Greene T,Kopple J, Lea J, Lewis J, Lipkowitz M, Miller P, 58. Aloisio KM, Swanson SA, Micali N, Field A, Horton NJ: Analysis of Richardson A, Rostand S, Wang X, Appel LJ: Baseline predictors of partially observed clustered data using generalized estimating renal disease progression in the African American Study of hyper- equations and multiple imputation. Stata J 14: 863–883, 2014 tension and kidney disease. J Am Soc Nephrol 17: 2928–2936, 2006 59. Birhanu T, Molenberghs G, Sotto C, Kenward MG: Doubly robust 41. Reed B, Helal I, McFann K, Wang W, Yan XD, Schrier RW: The and multiple-imputation-based generalized estimating equa- impact of type II diabetes mellitus in patients with autosomal tions. JBiopharmStat21: 202–225, 2011 dominant polycystic kidney disease. NephrolDialTransplant 60. DeSouza CM, Legedza AT, Sankoh AJ: An overview of practical 27: 2862–2865, 2012 approaches for handling missing data in clinical trials. J Biopharm 42. Palmas W, Pickering T, Eimicke JP, Moran A, Teresi J, Schwartz JE, Stat 19: 1055–1073, 2009 Field L, Weinstock RS, Shea S: Value of ambulatory arterial 61. Lachin JM: Fallacies of last observation carried forward analyses. stiffness index and 24-h pulse pressure to predict progression of Clin Trials 13: 161–168, 2016 albuminuria in elderly people with diabetes mellitus. Am J Hy- 62. Shepherd J, Breazna A, Deedwania PC, LaRosa JC, Wenger NK, pertens 20: 493–500, 2007 Messig M, Wilson DJ; Treating to New Targets Steering Committee 43. Gardner LI, Holmberg SD, Williamson JM, Szczech LA, Carpenter and Investigators: Relation between change in renal function and CC, Rompalo AM, Schuman P, Klein RS; HIV Epidemiology Re- cardiovascular outcomes in atorvastatin-treated patients (from search Study Group: Development of proteinuria or elevated the Treating to New Targets [TNT] Study). Am J Cardiol 117: 1199– serum creatinine and mortality in HIV-infected women. J Acquir 1205, 2016 63. Misra M, Vonesh E, Churchill DN, Moore HL, Van Stone JC, Nolph Immune Defic Syndr 32: 203–209, 2003 KD: Preservation of glomerular filtration rate on dialysis 44. Mills KT, Chen J, Yang W, Appel LJ, Kusek JW, Alper A, when adjusted for patient dropout. Kidney Int 57: 691–696, 2000 Delafontaine P,Keane MG, Mohler E, Ojo A, Rahman M, Ricardo 64. Liu C, Ratcliffe SJ, Guo W: A random pattern mixture model for AC, Soliman EZ, Steigerwalt S, Townsend R, He J; Chronic Renal ordinal outcomes with informative dropouts. Stat Med 34: 2391– InsufficiencyCohort(CRIC)StudyInvestigators: Sodiumexcretion 2402, 2015 and the risk of cardiovascular disease in patients with chronic 65. Michiels B, Molenberghs G, Bijnens L, Vangeneugden T, Thijs H: kidney disease. JAMA 315: 2200–2210, 2016 Selection models and pattern-mixture models to analyse longi- 45. Lemley KV, Boothroyd DB, Blouch KL, Nelson RG, Jones LI, tudinal quality of life data subject to drop-out. Stat Med 21: Olshen RA, Myers BD: Modeling GFR trajectories in diabetic 1023–1041, 2002 nephropathy. Am J Physiol Renal Physiol 289: F863–F870, 2005 66. Ten H, Reboussin BA, Miller ME, Kunselman A: Mixed effects 46. FanL,TighiouartH,LeveyAS, BeckGJ,SarnakMJ: Urinarysodium logistic regression models for multiple longitudinal binary excretion and kidney failure in nondiabetic chronic kidney dis- functional limitation responses with informative drop-out and ease. Kidney Int 86: 582–588, 2014 confounding by baseline outcomes. Biometrics 58: 137–144, 47. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro 3rd AF,Feldman 2002 HI, Kusek JW, Eggers P, Van Lente F, Greene T, Coresh J; CKD-EPI 67. Song X, Davidian M, Tsiatis AA: A semiparametric likelihood (Chronic Kidney Disease Epidemiology Collaboration): A new approach to joint modeling of longitudinal and time-to-event equation to estimate glomerular filtration rate. AnnInternMed150: data. Biometrics 58: 742–753, 2002 604–612, 2009 68. Tsiatis AA, Davidian M: Joint modeling of longitudinal and time- 48. Brunner-La Rocca HP,Knackstedt C, Eurlings L, Rolny V,Krause F, to-event data: An overview. Stat Sin 14: 809–834, 2004 ‐ Pfisterer ME, Tobler D, Rickenbacher P, Maeder MT; TIME CHF 69. Fitzmaurice G, Molenberghs G, Lipsitz SR: Regression models for investigators: Impact of worsening renal function related to longitudinal binary responses with informative drop-outs. J R Stat medication in heart failure. Eur J Heart Fail 17: 159–168, 2015 Soc Series B Stat Methodol 57: 691–704, 1995 49. Filipozzi P, Ayav C, Ngueyon Sime W, Laurain E, Kessler M, 70. Vonesh EF, Greene T, Schluchter MD: Shared parameter models Brunaud L, Frimat L: Trajectories of CKD-MBD biochemical pa- for the joint analysis of longitudinal data and event times. Stat Med rameters over a 2-year period following diagnosis of secondary 25: 143–163, 2006 hyperparathyroidism: A pharmacoepidemiological study. BMJ 71. Perkins RM, Bucaloiu ID, Kirchner HL, Ashouian N, Hartle JE, Yahya Open 7: e011482, 2017 T: GFR decline and mortality risk among patients with chronic 50. Nagin DS, Odgers CL: Group-based trajectory modeling in kidney disease. Clin J Am Soc Nephrol 6: 1879–1886, 2011 clinical research. Annu Rev Clin Psychol 6: 109–138, 2010 51. Franklin JM, Shrank WH, Pakes J, Sanfe´lix-Gimeno G, Matlin OS, Brennan TA, Choudhry NK: Group-based trajectory models: A Published online ahead of print. Publication date available at www. new approach to classifying and predicting long-term medication cjasn.org. adherence. Med Care 51: 789–796, 2013 52. Schluchter MD, Greene T,Beck GJ: Analysis of change in the presence This article contains supplementalmaterialonlineathttp://jasn. of informative censoring: Application to a longitudinal clinical trial of asnjournals.org/lookup/suppl/doi:10.2215/CJN.11311116/-/ progressive renal disease. Stat Med 20: 989–1007, 2001 DCSupplemental.