Analyzing Spatial Longitudinal Incidence Patterns Using Dynamic Multivariate Poisson Models
Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University
By
Yihan Sui, B.S., M.S.
Graduate Program in Biostatistics
The Ohio State University
2018
Dissertation Committee:
Dr. Grzegorz A. Rempala, Advisor
Dr. Chi (Chuck) Song, Advisor
Dr. Oksana Chkrebtii c Copyright by
Yihan Sui
2018 Abstract
Methods of multivariate analysis for continuous data have been applied extensively in epidemiology, economics, engineering, and other fields. However, multivariate models for count data have not been applied to a similar extent, and there is relatively little work published on models, especially when accounting for both spatial and temporal dependence. In the first part of the thesis, we propose a hierarchical multivariate
Poisson (MVP) model that simultaneously models spatial and temporal correlation of observed data counts in a fairly general setting. In particular, MVP allows for modeling the spatially/temporally dependent counts as a function of location-specific and time-varying covariates. To characterize temporal trends, we propose a broken- line regression model within MVP and apply it to joinpoint detection. Bayesian inference is conducted using Markov chain Monte Carlo (MCMC) methods to ap- proximate required posterior summaries. We use a backward selection algorithm coupled with the shaver method to obtain joinpoint coefficients via Bayesian Lasso.
We apply the proposed model to spatial temporal pertussis incidence data from 2000 to 2015 from several midwestern states in the U.S. To evaluate the appropriateness of the model, in the second part of this thesis we develop a goodness-of-fit (GOF)
ii statistic for fitting discrete generalized linear models (GLMs) based on the sum of standardized residuals (SSRs). This work is an extension of our earlier work (Chen L. et al. [1]) which proposed a GOF test for binary responses. We derive the asymptotic distribution of the test statistic and show how it can be applied to popular count regression models such as Poisson regression, Negative Binomial regression and Bino- mial regression using different link functions. Using numeric examples we show that the proposed test is substantially more powerful than most of the currently available
GOF tests under various model misspecification scenarios applied to various discrete
GLMs like logistic, Poisson, and negative binomial.
iii Acknowledgments
First, I would like to thank my wonderful parents, Daifu and Guiping for their love, trust and encouragement along the way. I would not be where I am today were it not for their love and guidance.
I am grateful to my co-advisors, Professor Grzegorz A. Rempala and Professor Chi
Song, both of whom have devoted a substantial amount of time and effort in train- ing me to be a better researcher over the past few years. Without their guidance and persistent help this work would not have been possible. Professor Grzegorz A.
Rempala is the reason I first became interested in multivariate count modeling, and since then I have thoroughly enjoyed working in this area with him. He has pro- vided me with numerous opportunities to learn and grow through research, teaching, mentoring, and collaboration. I also would like to express appreciation to Professor
Chi Song. He is a patient mentor and one of the smartest people I know. Some of my fondest memories in graduate school were hours of research discussions with him in his office. He has also become a good friend, whom I admire professionally and personally.
iv I thank the other member of my dissertation committee − Professor Oksana Chkrebtii, and the member of my candidacy committee − Professor Kellie J. Archer for their time and effort in evaluating my work and providing with me feedbacks.
Finally, I would also like to thank all the professors and staffs in the program of biostatistics who have provided guidance and support during my time at The Ohio
State University.
v Vita
June 5, 1990 ...... Born - Jilin, China
2013 ...... B.S. Biological Science
2015 ...... M.S. Statistics
2013-present ...... Graduate Associate, The Ohio State University.
Publications
Research Publications
Chen Lu, Sui Yihan, Song Chi, Grzegorz A. Rempala (2017). The Sum of Stan- dardized Residuals: Goodness of Fit Test for Binary Response Model Statistics in Medicine, doi: 10.1002/sim.7644.
Fields of Study
Major Field: Biostatistics
vi Table of Contents
Page
Abstract ...... ii
Acknowledgments ...... iv
Vita...... vi
List of Tables ...... x
List of Figures ...... xii
1. Introduction ...... 1
1.1 Mathematical Modeling of Epidemics ...... 1 1.1.1 Multivariate Models ...... 3 1.1.2 Multivariate Poisson (MVP) Distribution ...... 4 1.2 Generalized Linear Models (GLMs) ...... 6 1.2.1 Estimating Equations ...... 7 1.2.2 Goodness of Fit (GOF) Test for GLM ...... 8 1.3 Thesis Outline ...... 8
2. Dynamic Multivariate Poisson (DMVP) Model ...... 11
2.1 Modeling Framework ...... 11 2.1.1 Non-Dynamic Multivariate Poisson Model ...... 12 2.1.2 Dynamic Multivariate Poisson (DMVP) Model ...... 14 2.2 Multivariate Poisson Joinpoint Model ...... 17
vii 2.3 Bayesian Estimation ...... 21 2.3.1 The Posterior Distribution for AR(1) Model ...... 22 2.3.2 The Posterior Distribution for ARIMA(1,1,0) Model . . . . 24 2.3.3 The Estimation of Joinpoint Parameters b ...... 27 2.3.4 The update of Y and ξ ...... 28 2.4 Simulation Studies ...... 29 2.5 Analysis of Pertussis Data ...... 32 2.5.1 Background ...... 33 2.5.2 Modern Resurgence of Pertussis ...... 36 2.5.3 Modeling Pertussis Trends ...... 38 2.5.4 Analysis Details ...... 39 2.5.5 Conclusion ...... 45
3. The Goodness of Fit (GOF) in Generalized Linear Model (GLM) . . . . 50
3.1 Summary of Previous Research on GOF ...... 50 3.1.1 GOF Tests for Binary Outcomes ...... 52 3.1.2 GOF Tests for Discrete Outcomes ...... 56 3.2 Proposed Test Statistic - Standardized Residuals ...... 59 3.3 Numerical Examples ...... 66 3.3.1 Examples for Binary Variables ...... 67 3.3.2 Poisson Regression Example 1 ...... 75 3.3.3 Poisson Regression Example 2 ...... 78 3.3.4 Negative Binomial Example 1 ...... 79 3.3.5 Negative Binomial Example 2 ...... 82 3.4 Model Selection Example ...... 86 3.4.1 Model Selection Example for Binary Variable ...... 86 3.4.2 Model Discrimination Example for Count Variable . . . . . 87 3.5 Discussion ...... 89
4. Future Work ...... 91
Bibliography ...... 94
Appendices 103
viii A. The Sum of Standardized Residuals: Goodness of Fit Test for Binary Response Model ...... 103
ix List of Tables
Table Page
2.1 Simulation results for parameter β and γ in P = 1, Q = 1 ...... 30
2.2 Simulation results for parameter β and γ in P = 2,Q = 1 ...... 31
2.3 History of pertussis vaccines ...... 35
2.4 History of acellular pertussis vaccines ...... 36
2.5 Pertussis Vaccination Coverage (%) ...... 48
3.1 Selected variables description in the Boston Marathon data ...... 68
3.2 Proportion H0 rejected at the α = 0.05 using sample size of 445 with 10000 replications in quadratic models using the Boston Marathon data 69
3.3 Selected variables description in Cleveland data ...... 71
3.4 Proportion H0 rejected at the α = 0.05 using sample size of 303 with 10000 replications in interaction models using the Cleveland data . . 72
3.5 Selected variables description in NHANES data ...... 73
3.6 Proportion H0 rejected at the α = 0.05 using sample size of 373 with 10000 replications in interaction models using the NHANES data . . 74
x 3.7 Selected variables description in the Affairs data (n=601) ...... 76
3.8 Proportion H0 rejected at the α = 0.05 using sample size of 601 with 10000 replications using the Affairs data ...... 76
3.9 Selected variables description in the German Credit data (n=1,000) . 80
3.10 Proportion H0 rejected at the α = 0.05 using sample size of 1000 with 10000 replications using the German Credit data ...... 80
3.11 Selected variables description in the Bad Health data ...... 83
3.12 Proportion H0 rejected at the α = 0.05 using sample size of 1127 with 10000 replications using the Bad Health data ...... 83
3.13 Selected variables description in the Fishing data ...... 85
3.14 Proportion H0 rejected at the α = 0.05 using sample size of 147 with 10000 replications in sqrt link model using the Fishing data ...... 86
3.15 Selected variables description in HINTS data ...... 87
3.16 The results of Cn and its p-values under each model using the HINTS data ...... 88
3.17 Pertussis data ...... 89
3.18 The results of Cn and its p−values under Poisson and negative bimo- nial model using Pertussis data ...... 89
xi List of Figures
Figure Page
1.1 An overview of mathematical models for infectious diseases generated by Siettos and Russo[2] ...... 3
2.1 Group sampling illustration ...... 28
2.2 Pertussis Incidence by Age Group ...... 37
2.3 Pertussis Incidence Cases ...... 38
2.4 Pertussis Rate(per 100,000) 2008 ...... 39
2.5 Pertussis Rate(per 100,000) 2009 ...... 40
2.6 Pertussis Rate(per 100,000) 2010 ...... 41
2.7 Pertussis Rate(per 100,000) 2011 ...... 42
2.8 Pertussis Rate(per 100,000) 2012 ...... 43
2.9 Pertussis Rate(per 100,000) 2013 ...... 44
2.10 Pertussis Rate(per 100,000) 2014 ...... 45
2.11 Pertussis Rate(per 100,000) 2015 ...... 46
xii 2.12 Pertussis Rate(per 100,000) 2016 ...... 47
2.13 9 States ...... 47
2.14 DMVP and Simplified Poisson Model Fitting ...... 49
3.1 Proportion H0 rejected at the α = 0.05 using sample size of 601 with 10000 replications in interaction models using the Affair data . . . . . 77
3.2 Proportion H0 rejected at the α = 0.05 over increase sample size with 10000 replications using the German Credit data ...... 81
3.3 Proportion H0 rejected at the α = 0.05 in quadratic models with sample size of 1127 with 10000 replications in the Bad Health data . 84
xiii Chapter 1: Introduction
1.1 Mathematical Modeling of Epidemics
The usage of mathematical modeling methodologies in epidemics dates back to 1760 when Daniel Bernoulli studied the outbreak of smallpox in England[3, 4]. Recently, dynamical systems modeling methods have become popular in epidemiological re- search, mostly due to their ability to accommodate social, economic, and demo- graphic factors [2]. In particular, the modeling with multivariate count data has become increasingly prevalent in big data applications in many different domains.
Such data are usually dependent and with both positive or negative correlations present. For instance, in the public health domain, the Centers for Disease Control and Prevention (CDC) provides the database of national-wide disease outbreaks by year, region, and disease type [5]. The epidemiological trends such as disease inci- dence data are monitored and displayed on the CDC website with graphics, maps and are also provided in text-file formats. For example, reported historic pertussis cases data is available on the CDC website [6]. These disease data have been collected
1 over past 20 years from all the states through their respective health departments via the National Notifiable Diseases Surveillance System (NNDSS). For epidemiologists
NNDSS data are invaluable for understanding the trends and the spreads of various communicable diseases, and for examining the underlying social, economic and de- mographic factors that are crucial to to develop better decision-making policies and to help better prepare for pandemic outbreaks in the US [7].
Spatial variation in disease incidence should be considered while modeling due to the ecological processes, such as pathogen dispersal might be clustered or infected hosts might be less likely to be on a long trip [8]. Moreover, these spatial distri- butions of infectious diseases often change through time [8]. In order to study the disease incidence and its changes over time, we need to establish convenient modeling methods that can take into account the spatial and temporal relationship among the incident rates. Siettos and Russo [4] suggested that the mathematical approaches used in epidemiology can be divided into 3 categories (presented in Fig. 1.1): 1)
Statistical approaches such as regression within the context of spatial patterns to model disease epidemics; 2) Mathematical models of dynamical systems(mechanistic state-space models) such as the susceptible , infected, and recovered (SIR) model;
3) Machine learning techniques using data from internet platforms such as Google
Flu. In this thesis we focus on the first category. We approach the problem at the population level by adopting a statical multivariate Poisson model, expanding it into a hierarchical model, and extending the latter to analyze spatial temporal data.
2 Figure 1.1: An overview of mathematical models for infectious diseases generated by Siettos and Russo[2]
1.1.1 Multivariate Models
The use of the multivariate normal (MVN) distribution has a long history for the analysis of continuous data because of its convenient statistical properties. MVN distribution has been well established tool to model spatial dependencies due to
T its convenient statistical properties [9]. Let X = (X1, ..., Xn) be a vector of ran- dom variables. The mean vector and the variance-covariance matrix are denoted
T n by µ = (µ1, ..., µn) (µ ∈ R ) and Σn×n (Σn×n is a positive semi-definite matrix), respectively. The desirable properties of MVP distribution are: 1) their explicit
1 1 1 joint probability density function exists: fX (x|µ, Σ) = (2π)n/2 (det)1/2(Σ) exp − 2 (x − µ)T Σ−1(x − µ) , where x is a realization of the random vector X (x ∈ Rn) and 2)
3 the covariance function (Σ) accurately specifies the correlation structure. In contrast, these properties are not immediately available for any multivariate count distribu- tion. There has been relatively few works on implementing multivariate count model to deal with discrete spatial data as discussed in Section 1.1.2. Here we aim to con- struct a multi-dimensional discrete distribution with Poisson margins that allows for simple description of the dependencies among variables.
1.1.2 Multivariate Poisson (MVP) Distribution
One of the most commonly accepted definition of MVP distribution is given by
T Mahamunulu et al. [10]: a m-variate vector X = (X1, ..., Xm) is said to follow the
T MVP distribution through a vector of hidden variables Y = (Y1, ..., Yk) , such that
X = AY . This definition is based on a projection Nk → Nm, k ≥ m. Here, the
T hidden variables Y = (Y1, ..., Yk) independently follow a Poisson distribution. A is a m × k matrix of values 0 and 1 that determines the correlation structure of X.
This is the distribution that our dynamic MVP model is based on. Our dynamic
MVP borrows the idea of hidden variable decomposition scheme to help construct the variance-covariance structure. Detailed description of the model will be provided in Chapter 2.
The formulation of multivariate Poisson distribution dates back to 1925 when differ- ential equations were used to derive a bivariate Poisson distribution [11, 12]. A more understandable but equivalent formulation to derive the bivariate Poisson distribu- tion is to use the summation of independent Poisson variables [12]. The derivation
4 of a special case of multivariate Poisson model were presented recently by Tsionas et al. [13, 14] and Karlis [15] using a shared component construction. The limitation of this model is that it assumes the correlation between all pairs of variables being the same by using a single common covariance term. In 2005, this model was extended by Karlis and Meligkotsidou [16] to allow for a more flexible covariance structure among variables by permitting different correlations between pairs of variables. This extension of the covariance structure expanded the model’s application into a num- ber of new fields [16]. For example, in epidemiology it is of great interest to study the covariance structure of the diseases incidence in a given area from a geographical perspective. Although the model of Karlis and Meligkotsidou [16] can capture the spatial correlation, it is not a dynamic model and fails to account for the temporal trends. The research on the extension of the MVP for modeling longitudinal data is relatively sparse. Pedeli et al.[17] introduced a bivariate integer-valued autore- gressive process of order 1 (BINAR(1)) and applied it to a bivariate temporal data concerning daytime and nighttime road accidents in Netherlands. They focused on bivariate Poisson and bivariate negative binomial distributions. Aktekin et al. [18] developed a MVP model they referred to as multivariate Poisson-scaled beta (MPSB) that allows for temporal dependency in the counts as well as dependency across mul- tiple series. They then applied MPSB to web traffic analysis using weekly consumer demand data.
5 In the following chapters, we construct a dynamic MVP that can be applied to lon- gitudinal (time series) data based on the MVP derived by Karlis and Meligkotsidou
[16]. We then allow for the temporal trends detection by adding a piecewise regres- sion component. Explanatory variables can be included in the model. Inference is performed via the Metropolis-Hastings within Gibbs sampler. We adopt the adaptive rejection sampling (ARS) scheme to improve the algorithm efficiency.
1.2 Generalized Linear Models (GLMs)
Generalized linear models (GLMs) are a popular class of regression models that generalize linear regression to accommodate a variety of outcome variable types such as discrete, binary, and categorical [19]. In what follows, we borrow the notation
T from McCullagh and Nelder [20]. The sample Y = (Y1, ..., Yk) is assumed to have independent distribution of the natural exponential family (NEF) such that each Yi has the p.d.f.
n(y θ − b(θ ))o f (y; η, φ) = exp i i i c(y ; φ), i = 1, ...n. (1.1) Yi α(φ) i where n is the number of samples, αi(·), b(·) and c(·) are known functions, and θi is a canonical(natural) parameter for some known dispersion function α(φ) with dispersion parameter φ. The mean and variance of Yi are given by
( 0 E(Yi) = µ = b (θi), 00 Var(Yi) = V (µi) = b (θi)α(φ), i = 1, ...n.
There are three components in GLM:
6 1. Random Component - refers to the variation of the response variable Y due
to its probability drawn from the density function; e.g. in Poisson regression
models, Y has Poisson distribution.
T 2. Systematic Component - refers to the set of covariates Zi = (Zi1,Zi2, ...ZiP ) ;
T e.g. θi = β Zi in Poisson regression.
3. Link Function - refers to the link function g between the mean of the random
and the systematic components. The natural parameter θi is a function of the
T T mean θi = g(µi), such as g(µi) = β Zi, that is, the link between θi and β Zi is identity.
1.2.1 Estimating Equations
T Let Zi denote a p-vector of covariates for the ith individual. From item 3 in the
T previous section, the mean function is assumed to relate to Zi β through the link function
T g(µ(θi)) = Zi β, i = 1, ..., n, where g is the link function and β is a p-vector of parameters. As discussed above,
−1 T −1 if µ = g and θi = Zi β, then g is the canonical link. Now define ψ = (g ◦ µ) , then the p.d.f can be written as
ψ(ZT β)y − b(ψ(ZT β)) f (y; η, φ) = exp{ i i i }c(y ; φ), (1.2) Yi α(φ) i
7 and thus the log-likelihood function is
n T T X h ψ(Z β)yi − b(ψ(Z β))i log`(β) = logc(y ; φ ) + i i . i i α(φ) i=1 The MLE of β can be obtained by solving the likelihood score equations
n ∂log`(β) 1 X n 0 o = y − µ(ψ(ZT β))ψ (ZT β)Z = 0. ∂β α(φ) i i i i i=1 0 T In this work, we study primarily the case when ψ (Zi β) is not proportional to √ 1 . This is the case for most discrete GLM’s. V (µi)
1.2.2 Goodness of Fit (GOF) Test for GLM
The issue of measuring discrepancy between model predicted and observed values arises once model is estimated from the data. This concept of a model goodness of fit
(GOF) helps quantify this discrepancy by considering the (signed) distance between observations and their predicted mean values under the specific link function and the set of covariates (e.g., linear form, quadratic form and interactions). Measurement of GOF is crucial to evaluate whether the model explains the data sufficiently well.
Motivated by our longitudinal count data problem we consider here a new GOF measure that appears to be well suited for discrete GLMs.
1.3 Thesis Outline
The thesis is organized into 3 chapters. Beyond this introductory Chapter 1, Chapter
2 will introduce the modeling framework of dynamic multivariate Poisson (DMVP)
8 model starting from a static MVP distribution and its extension to dynamic data over time. Section 2.1.1 introduces different decomposition schemes accounting for spatial dependency of multivariate count variables. Section 2.1.2 discusses the construction of DMVP model for a given time series. We then consider a joinpoint model in
Section 2.2 to identify diseases trends and to detect changes. A shaver methods is employed in Section 2.2 for parameter estimation. We then further discuss the details of the parameter estimation from the Bayesian prospective in Section 2.3.
Simulation studies are performed to evaluate the model performance in Section 2.4.
As an example, a real application using pertussis dataset is shown in Section 2.5 to demonstrate the advantages and disadvantages of DMVP model over simple Poisson regression model. In Chapter 3 we propose a GOF statistic Cn for the discrete GLMs based on the sum of standardized residuals (SSRs). Section 3.1 will review currently used GOF tests that are built into statistical software for exponential modeling families. The derivation of the SSRs test statistic Cn and its asymptotic properties are discussed in Section 3.2. Next, via numerical examples we demonstrate that
GOF test based on Cn outperforms many of the existing more complicated currently used methods. We also adopt real data examples in simulation studies to investigate the power of the proposed test and to compare it Section 3.3 with existing methods in logistic, Poisson, and negative binomial regressions. Finally, a simple illustration of Cn statistic usage in the model selection problem is presented in Section 3.4. We
9 discuss limitations of the test statistics in Section 3.5 where we also outline some possible future research directions.
10 Chapter 2: Dynamic Multivariate Poisson (DMVP) Model
2.1 Modeling Framework
In this section, we will first illustrate the definition of MVP distribution introduced by Karlis et al. [16] with examples in simple settings. Then we will extend the MVP by introducing the temporal component to capture trends in dynamic data.
The MVP distribution is derived from a multivariate reduction scheme (Karlis et al.)
T [16]. Let X = (X1,X2, ..., Xm) denote the observed m-vector of variables at location
1, ..., m, where m is the total number of locations. Each element Xi represents the outcome variable such as the total number of infected at the specific location i, i ∈ {1, 2, ..., m}. X is said to follow a MVP distribution if X = AY , where the
T components of the vector Y = (Y1,Y2, ..., Yk) are hidden variables that are assumed to have independent Poisson distributions, ie. Yi ∼ Poi(θi), i ∈ {1, 2, ..., k} (k is the total number of the hidden variables in Y ). A is an m × k matrix (m ≤ k) with 0 and 1 elements that determines the correlation structure of X . The variability of
X is explained through the variability of k independent univariate Poisson random
11 variables Y1,Y2, ..., Yk. As discussed later, the elements in Y can be considered as main effects and covariance effects. The mean and variance-covariance matrix of X are
E(X ) = AE(Y ) = Aθ, and
V ar(X) = AΣAT , (2.1)
T where vector θ = (θ1, θ2, ..., θk) is the mean of Y , and Σ = diag(θ1, θ2, ...θk) is the variance-covariance matrix of Y . Because the summation of independent Poisson random variables still follows Poisson distribution, we can derive that each element of X follows a univariate Poisson distribution marginally [17, 21]. An example of the MVP distribution will be presented in the next section.
2.1.1 Non-Dynamic Multivariate Poisson Model
We now will provide illustrative examples of MVP distribution described above us- ing a two-way covariance setting where only correlations between two locations are considered. Though the general model could be more flexible to account for more complex correlation structure such as correlation among 3 locations, we only consider a simplified model here.
A two-way covariance structure MVP model is derived by decomposing matrix A into two parts. The first part of A is an identity matrix that formulates the main effect of the outcome variables. The second part of A induces the covariance structure of the
12 outcome variables. Note that a different correlation term for each pair of variables is allowed in this model, such that the parameters of the MVP model have meaningful
T interpretation. We now consider the hidden variables Y = (Y1,Y2, ..., Yk) consisting of two parts as in matrix A, the main components and the covariance components:
T T (Y1,Y2, ..., Yk) = (Y1,Y2, ..., Ym, ξ1,2, ξ1,3, ..., ξm−1,m)
T Main components are the m out of k elements in Y = (Y1,Y2, ..., Yk) represent- ing the variation that can be explained by the location-specific covariates. Whereas
T ξ = (ξ1,2, ξ1,3, ..., ξm−1,m) are the covariance (pairwise) effect components that can be modeled by explanatory variables representing the strength of the correlation between locations. Note that, when the number of main effect elements is m, the
m number of covariance effect elements is 2 . The distributions of ξ are also inde- pendent Poissons, that is, ξi,j ∼ Poi(θi,j), where 1 ≤ i ≤ j ≤ m. The same for
Yi ∼ Poi(µi), where i ∈ {1, 2, ..., m}. An example of two-way covariance structure is the trivariate Poisson model, with m = 3 of the form
X1 = Y1 + ξ1,2 + ξ1,3,
X2 = Y2 + ξ1,2 + ξ2,3, (2.2)
X3 = Y3 + ξ1,3 + ξ2,3, where matrix A is
1 0 0 1 1 0 A = 0 1 0 1 0 1 . (2.3) 0 0 1 0 1 1
13 T In general without considering time index, we have X = (X1,X2, ..., Xm) follows
MVP distribution such that X Xi = Yi + ξi,j, (2.4) j6=i where ξi,j = ξj,i.
2.1.2 Dynamic Multivariate Poisson (DMVP) Model
In practice, the infectious disease data are often collected over a long period of time to study the long-term trends in disease incidence. Therefore, we need to extend the
MVP distribution to account for temporal effect by incorporating parameter evolu- tion described by a time series model. The idea of applying time series approaches to generalized linear models (GLMs) was originally introduced by Gamerman et al.
[22] in the context of dynamic hierarchical models in order to account for the tem- poral trend. The dynamic model comprises two parts: observation equations that describe the distribution of the observations, and system equations that describe the evolution of the parameters through time. For location i at time t, these equations can be written as follows.
Observation equations: X Xi,t = Yi,t + ξi,j,t, j6=i (2.5) Yi,t ∼ Poi(µi,t); log(µi,t) = βi,t,
ξi,j,t ∼ Poi(θi,j,t); log(θi,j,t) = γi,j,t, 1 ≤ i 14 System equations: (β) 2 βi,t = Gt βi,t− + i,t, i,t ∼ N(0, σ ), (2.6) (γ) 2 γi,j,t = Gt γi,t− + εi,j,t, εi,j,t ∼ N(0, v ). (β) (β) Here Gt and Gt are known matrices that formulate βi,t and γi,j,t through a linear combination of the parameters prior to time t. In this work, we adopt simple AR(1) model and ARIMA(1, 1, 0) model for β and γ. These are discussed in Section 2.3. βi,t− and γi,j,t− are vectors that include all elements prior to time t. and ε are disturbance terms which are assumed to have independent Gaussian distributions with mean zero and variance σ2 and v2 respectively; t ∈ {1, 2, ..., T }, with T is total time points. In order to further extend the applicability of the dynamic model, we incorporate the time-varying explanatory variables into the DMVP. This can be achieved by introducing log-link function between the explanatory variables and the parameters of interest, similarly as in the simple Poisson regression model. The proposed model can be written using matrix notation as follows. Observation equations: Xt = Y t + ξt, t ∈ {1, 2, ..., T }, T Y t ∼ Poi(µt), log(µt) = Zt βt, (2.7) T ξt ∼ Poi(θt), log(θt) = Dt γt. System equations: (β) βt = Gt βt− + t, t ∼ N(0, W ), (2.8) (γ) γt = Gt γt− + εt, εt ∼ N(0, V ), 15 T where Xt = (X1,t,X2,t, ..., Xm,t) is the vector of outcome variables − the number of T T infections for location 1, ..., m at time t. Y t = (Y1,t, ...Ym,t) , ξt = (ξ1,2,t, ..., ξ(m−1),m,t) are the hidden variables representing the mean and covariance components at time t, that follow independent Poisson distributions. Zt is a P × m matrix of explanatory variables for main effect components Y t at time t. Note that Zt are location-specific variables that help explain the disease incidence rate, such as immunity, medical facility and weather conditions. Here P is the number of explanatory variables avail- m able in Zt. Dt is a Q × 2 matrix of explanatory variables for covariance effect components ξt at time t. Note that Dt are variables that measure the correlation strength between locations, such as passengers travel volume between location i and j, or simply the distance between location i and j. Here Q is the number of covariates available for covariance effect. βt and γt are vectors of the regression coefficients for (β) (γ) covariates ξt and Zt respectively. Gt and Gt are known matrices of the evolu- tion equations for βt and γt respectively, such as AR(1) model and ARIMA(1, 1, 0) model. W and V are variance-covariance matrices that do not vary over time. The model can also be written as follows. Observation equation: X Xi,t = Yi,t + ξi,j,t, j6=i P X Yi,t ∼ Poi(µi,t), log(ui,t) = Zi,p,tβp,t, (2.9) p=1 Q X ξi,j,t ∼ Poi(θi,j,t), log(θi,j,t) = Di,j,qγq,t, q=1 16 System equation: (β) 2 βp,t = Gt βp,t− + p,t, p,t ∼ N(0, σ ), (2.10) (γ) 2 γq,t = Gt γq,t− + εq,t, εq,t ∼ N(0, v ). 2.2 Multivariate Poisson Joinpoint Model Joinpoint modeling in regression analysis arises when data sets are broken down into different blocks in such way that the observations within each block follow the same regression model, but observations from different blocks follow different models [23]. It is essential to find proper functional form of regression parameters within block. Typically two types of joinpoints are considered, the instantaneous and gradual [24]. The patterns in gradual change point model are usually implicit and are character- ized with the help of piece-wise linear models [24]. Gradual change point models have been commonly used in public health applications. For example, Czajkowski et al. [25] extended the use of a broken-line model and applied it to a longitudi- nal cancer mortality data to search for the number and location of trend changes. Campos et al. [26] used segmented models of Poisson regression to analyze chronic obstructive pulmonary disease (COPD) mortality trends by sex in 27 countries in the European Union (EU), for the period of 1994 to 2010, whereas Malvezzi et al.[27, 28] estimated numbers of deaths from cancers and worldwide age-standardized mortality rates (ASRs) in 2013 using the World Health Organization mortality and population database, via logarithmic Poisson count data joinpoint models. Cassel [29] performed a piecewise logistic regression adjusting for age, sex, and race/ethnicity to examine 17 the trends in the prevalence of extreme obesity among US preschool-aged children living in low-income families from 1998 to 2010. The purpose of this current work is to propose a point detection method that allows us to identify the locations and number of the time joinpoints in spatial and longitudinal data. In other words, we would like to apply the broken-line model from Czajkowski et al. [25], and incorporate it into our DMVP model. Following the setting in Czajkowski et al. [25] we are focusing on analyzing the spatial discrete outcomes data collected from an infectious disease over a period of time. Recall that our observations are (X1,1, Z1,1, D1,2,1), ..., (Xm,T , Zm,T , Dm−1,m,T ), at time 1, ..., T that are ordered observation time points. Y1,1, ..., Ym,T are assumed to follow independent Poisson regressions with means µ1,1, ..., µm,T , and with unknown joinpoints with τ1 < ... < τw. Specifically we have w T X log(µi,t) = Zi,tβt + b0t + bl(t − τl)+, l=1 (2.11) where i = 1, .., m, t = 1, ...T, t+ = max{t, 0}, and βt is the P -variate vector of regression coefficients for the fixed main effect covariates at time t. b0 is the baseline th slope and bl is the slope change corresponding to the (l+1) segment with l = 1, ..., w and w is the total number of true joinpoints. τ1, ..., τw are the unknown joinpoints locations. Note that, model (2.11) reduces to a general GLM if the joinpoints are known. 18 When the number of joinpoints w is unknown, it needs to be estimated together with all other parameters. Using direct likelihood-based maximization for estimating the joinpoint parameters typically leads to over-estimation. This means that joinpoints are found at the maximum number of locations possible. Therefore, we need to add additional constrains to bound the maximum number of joinpoints. For instance, we can fit the model with penalties placed on b1, ..., bw to control for model complexity. For instance by adding a lasso penalty on the number of joinpoints will shrink the estimated parameter values towards 0: w X −n λ|bl|, l=1 (2.12) with some λ > 0. Here λ is referred to as the smoothing parameter. The lasso Pw penalty term n l=1 λ|bl| in (2.12) was introduced by Tibshirani et al. [30]. The lasso estimates are viewed as penalized least square estimates. The lasso estimates can be interpreted as the Bayesian posterior estimates where the regression coefficients are assumed to have independent and identical Laplace prior(i.e., double-exponential) [31]. Penalized likelihood with flat prior is used in model fitting here to obtain the posterior distribution of the estimates of change of slopes bs. A more detailed discussion of the joinpoints parameters estimation is provided in Section 2.3. Searching for the number of joinpoints requires us to re-estimate the parameters after each model update, which makes the problem of estimating the locations of change points of (2.11) computationally intensive. Gill et al.[24] proposed a computationally 19 simple and flexible shaver method for verifying the number of changes and approx- imating their locations. Shaver method starts off with an over-parametrized model with sufficient number of knots that are identified in order to ensure that the pre- specified joinpoints locations are close enough to the true knot locations. Then the over-parametrized model is fitted using the penalty function to obtain the regression coefficients and slopes. The extraneous knots are removed iteratively after the coef- ficients of the knots are estimated (remove the insignificant ones at each iteration). The details of this idea are presented below. We initiate the model by taking a sufficiently large number of knots τ1, ..., τh with h w over the time interval 1 − T , so that many of the slope bl may be estimated as zero. According to Gill et al. [24], shaver method proceeds as following: 1. Initiate with superfluous number of nodes h w across 1 − T . 2. Sample (βt, b, γt) from their posterior distributions described in Section 2.3. 3. Remove the insignificant nodes of which credible intervals for which parameter contain 0. 4. Re-estimate all coefficients (βt, b, γt). Note that we only estimate the coefficients (βt, γt) at the initial and last step to save computational time. The initial joinpoints h should not be chosen too large in order to circumvent the problem of unidentifiability. 20 2.3 Bayesian Estimation Our model fitting procedure is performed with the help of Bayesian methods. The description of our approach is based on the DMVP model discussed before with explanatory variables Z and D in a two-way covariance structure. The parame- ters we need to estimate are the regression coefficients β and γ, joinpoints slopes b = (b0, b1, ..., bw), and the hidden variables Y and ξ. More details are provided in Sections 2.3.1 − 2.3.4. The posterior distribution of regression parameters β and γ are obtained by using Adaptive Rejection Sampling (ARS) scheme. ARS is a powerful sampling technique for univariate log-concave probability density function that was proposed by Gilks et al. for Gibbs Sampling [32]. The sampling technique is adaptive, where it squeezes the rejection envelope to converge to its target density function very quickly and efficiently [32]. In 1995, Gilks et al. [33] generalized the ARS to include a Metropolis Hastings algorithm step for non-log-concave full conditionals. Since ARS helps to reduce the number of samples needed for MCMC convergence, it is therefore, used when the posterior distribution approximation is not conjugate and the density is computationally intensive. Recall that the system equation for the regression coefficients can be taken a variety of forms. We consider two cases AR(1) and ARIMA(1, 1, 0) that are discussed in Section 2.3.1 and 2.3.2 respectively. 21 Note that the density function of the observed data P (X|Z, D, β, γ, b) is not tractable. However, the density function of the hidden variables P (X|Z, β, b) and p(ξ|D, γ) are easy to handle, as they are both assumed to have independent Poisson distribu- tions. We assume that Y and ξ are conditionally independent given Z, D, β, b and γ. The full log-likelihood function is `(β, γ, b) T X = Poi Y t, ξt | βt, γt, Zt, Dt, b, λ t=1 T m w (2.13) X X T X = Poi yi,t; exp Zi,tβt + b0t + bl(t − τl)+ t=1 i=1 l=1 T X X T + Poi ξi,j,t; exp Di,jγt t=1 j 2.3.1 The Posterior Distribution for AR(1) Model We define the AR(1) evolution equations as 2 βp,t = βp,t−1 + p,t, p,t ∼ N(0, σp), p = 1, ..., P, 2 γq,t = γq,t−1 + εq,t, εq,t ∼ N(0, τq ), q = 1, ..., Q. (2.14) It is convenient to obtain the posterior distribution of β and γ in the AR(1) equation (2.14). Note that the sampling steps of the first and last parameters βp,1 and βp,T are different from the remaining ones. 22 In the case of t ∈ {2, ..., T − 1}, the posterior distribution of βp,t is: P (βp,t | X, Y , ξ, Z, D, γ, β−p,t, b, λ), p ∈ {0, ..., P } ∝ P (βp,t | Y t, Zt, β−p,t, b, λ) ∝ P (βp,t, Y t, Zt, β−p,t, b, λ) (2.15) ∝ P (Yt | Zt, βt, b, λ)P (βp,t | βp,t−1)P (βp,t+1 | βp,t) m Y = Poi(yi,t; µi,t)P (βp,t | βp,t−1)P (βp,t+1 | βp,t), i=1 where P (.)is the conditional density function, and the last two terms P (βp,t | βp,t−1)P (βp,t+1 | βp,t) follow the time series pattern in equation (2.14). In the case of t = 1, the posterior distribution of βp,t is: P (βp,1 | X, Y , ξ, Z, D, γ, β−p,1, b, λ), p ∈ {0, ..., P } ∝ P (βp,1 | Y 1, Z1, β−p,1, b, λ) ∝ P (βp,1, Y 1, Z1, β−p,1, b, λ) (2.16) ∝ P (Y1 | Z1, β1, b, λ)P (βp,2 | βp,1)π(βp,1) m Y = Poi(yi,1; µi,1)P (βp,2 | βp,1)π(βp,1). i=1 23 2 Here π(βp,1) denotes the prior distribution for βp,1(e.g., βp,1 ∼ N(βp,0, σ )). In the case of t = T , the posterior distribution of βp,T is P (βp,T | X, Y , ξ, Z, D, γ, β−p,T , b, λ), p ∈ {0, ..., P } ∝ P (βp,T | Y T , ZT , β−p,T , b, λ) ∝ P (βp,T , Y T , ZT , β−p,T , b, λ) (2.17) ∝ P (YT | ZT , βT , b, λ)P (βp,T | βp,T −1) m Y = Poi(yi,T ; µi,T )P (βp,T | βp,T −1). i=1 The calculation of posterior distribution of γ is analogous to β, except that Z is replaced by D. 2.3.2 The Posterior Distribution for ARIMA(1,1,0) Model AR(1) is the simplest time series model to account for temporal trend. However, depending on the data dynamic, a more flexible trend in time series might be needed. One can assume that β and γ in the system equation follow other time series (e.g., ARIMA sequences) with order (n1, n2, n3). An example of ARIMA(1, 1, 0) is given by 2 βp,t − βp,t−1 = κp(βp,t − βp,t−1) + p,t, p,t ∼ N(0, σp), p = 1, ..., P, 2 γq,t − γq,t−1 = ρd(γq,t − γq,t−1) + εq,t, εq,t ∼ N(0, vq ), q = 1, ..., Q. (2.18) Where κp and ρq are the moving average parameters for βp and γq respectively. Sampling from ARIMA(1, 1, 0) is similar to the AR(1) case discussed in section (2.3.1) 24 except that it involves more terms as estimating β and γ, since ARIMA has longer autoregressive lag. The estimation of β and γ are summarized in 3 parts. In the case of t ∈ {3, ..., T − 2}: P (βp,t | X, Y , ξ, Z, D, γ, ρ, κ, β−p,t, b, λ), p ∈ {1, ..., P } ∝ P (βp,t | Y t, Zt, β−p,t, b, λ, κp) ∝ P (βp,t, Y t, Zt, β−p,t, b, λ, κp) ∝ P (Yt | Zt, βt, b, λ)P (βp,t | βp,t−1, βp,t−2, κp) (2.19) P (βp,t+2 | βp,t+1, βp,t, κp)P (βp,t+1 | βp,t, βp,t−1, κp) m Y = Poi(yi,t; µi,t)P (βp,t+2 | βp,t+1, βp,t, κp)P (βp,t+1 | βp,t, βp,t−1, κp) i=1 P (βp,t | βp,t−1, βp,t−2, κp), where the last three terms P (βp,t+2 | βp,t+1, βp,t, κp)P (βp,t+1 | βp,t, βp,t−1, κp)P (βp,t | βp,t−1, βp,t−2, κp) follow the pattern in equation (2.18). In the case of t = T − 1 : P (βp,T −1 | X, Y , ξ, Z, D, γ, ρ, κ, β−p,T −1, b, λ), p ∈ {1, ..., P } ∝ P (βp,T −1 | Y T −1, ZT −1, β−p,T −1, b, λ, κp) ∝ P (βp,T −1, Y T −1, ZT −1, β−p,T −1, b, λ, κp) ∝ P (YT −1 | ZT −1, βT −1, b, λ)P (βp,T −1 | βp,T −2, βp,T −3, κp)P (βp,T | βp,T −1, βp,T −2, κp) m X = Poi(yi,T −1; µi,T −1)P (βp,T −1 | βp,T −2, βp,T −3, κp)P (βp,T | βp,T −1, βp,T −2, κp). i=1 (2.20) 25 Note that, it has one term less than the case t ∈ {3, ..., T − 2}. In the case of t = T : P (βp,T | X, Y , ξ, Z, D, γ, ρ, β−p,T , κ, b, λ), p ∈ {1, ..., P } ∝ P (βp,T | Y T , ZT , β−p,T , b, λ, κp) ∝ P (βp,T , Y T , ZT , β−p,T , b, λ, κp) (2.21) ∝ P (YT | ZT , βT , b, λ)P (βp,T | βp,T −1, βp,T −2, κp) m Y = Poi(yi,T ; µi,T )P (βp,T | βp,T −1, βp,T −2, κp). i=1 In the case of t = 2: P (βp,2 | X, Y , ξ, Z, D, γ, ρ, β−p,2, κ, b, λ), p ∈ {1, ..., P } ∝ P (βp,2 | Y 2, Z2, β−p,2, b, λ, κp) ∝ P (βp,2, Y 2, Z2, β−p,2, b, λ, κp) (2.22) ∝ P (Y2 | Z2, β2, b, λ)P (βp,4 | βp,3, βp,2, κp)P (βp,3 | βp,2, βp,1, κp)π(βp,2 | βp,1) m Y = Poi(yi,2; µi,2)P (βp,4 | βp,3, βp,2, κp)P (βp,3 | βp,2, βp,1, κp)π(βp,2), i=1 where π(βp,2) denotes the prior distribution for βp,2. Lastly, in the case of t = 1: P (βp,1 | X, Y , ξ, Z, D, γ, ρ, β−p,1, κ, b, λ), p ∈ {1, ..., P } ∝ P (βp,1 | Y 1, Z1, β−p,1, b, λ, κp) ∝ P (βp,1, Y 1, Z1, β−p,1, b, λ, κp) (2.23) ∝ P (Y1 | Z1, β1, b, λ)P (βp,3 | βp,2, βp,1, κp)π(βp,2 | βp,1)π(βp,1 | βp,0) m X = Poi(yi,1; µi,1)P (βp,3 | βp,2, βp,1, κp)π(βp,1), i=1 26 where π(βp,1) denotes the prior distribution for βp,1. The calculation of posterior distribution of γ is analogous to β, except that Z is replaced by D. 2.3.3 The Estimation of Joinpoint Parameters b Recall that b0 is the baseline slope and b1, ..., bh are unknown slope changes corre- sponding to knots τ1, ...τh that are assumed to be close enough to the true w joinpoint locations. h is the number of initial pre-specified knots such that h w. Note that, due to the fact that the values of all of the b0, ..., bh determine together the realiza- tion of the changing patterns over time. In the iterative fitting procedure, the slope change bl at knot l would impact all the slopes that appear later than knot l. This would make the Monte Carlo algorithm greatly inefficient. We therefore propose to perform group sampling of the slopes knot by knot so that in each iteration only the slope(s) connecting to the knot are updated, and all other slopes remain unchanged. Fig. 2.1 illustrates the sampling technique. There are 6 knots in this example at times integer 0-6 respectively and the corresponding slopes changes are b1 −b5. When the knot at time 1 is being updated, the corresponding slopes need to change are of segments 1 and 2 (6 segments are shown in Fig. 2.1), with other slopes keep the same. When it comes to the last knot (at time 6) being updated, only slope for segment 6 need to be updated, and all other slopes keep unchanged. 27 Figure 2.1: Group sampling illustration 2.3.4 The update of Y and ξ Note that, the hidden variables Yi,t and ξi,j,t cannot be updated one by one due to the constrain that they need to sum up to the outcome variables Xi,t. Therefore, sets of values {Yi,t,Yj,t, ξi,j,t, 1 ≤ i ≤ j ≤ m} are updated simultaneously. A discrete random [r−1] [r−1] [r−1] walk Metrpolis-Hastings algorithm is used to move from {Yi,t ,Yj,t , ξi,j,t , 1 ≤ th ∗[r] ∗[r] ∗[r] th i ≤ j ≤ m} at (r − 1) iteration to {Yi,t ,Yj,t , ξi,j,t, 1 ≤ i ≤ j ≤ m} at r iter- [r−1] [r−1] [r−1] ation, with transition density q(, ∗) ≡ Uniform L(ξi,j,t ),U(Yi,t ,Yj,t ) , where [r−1] [r−1] the lower bound L(ξi,j,t ) takes the value of max(−ξi,j,t , −M), and the upper [r−1] [r−1] [r−1] [r−1] bound U(Yi,t ,Yj,t ) takes the value of min(Yi,t ,Yj,t ), with M is tuning parameter. In other words, at each step, a random variable δ[r] is sampled from [r−1] [r−1] [r−1] Uniform L(ξi,j,t ),U(Yi,t ,Yj,t ) that the next iteration may become 28 ∗ [r−1] [r] Yi,t = Yi,t − δ , ∗ [r−1] [r] Yj,t = Yj,t − δ , (2.24) ∗ [r−1] [r] ξi,j,t = ξi,j,t + δ . f ∗ The acceptance probability of the algorithm is given by α(, ∗) = min(1, f ), where ∗ ∗ ∗ ∗ T f = Poi(yi,t; µi,t)Poi(yj,t; µj,t)Poi ξi,j,t; exp Di,jγt × (2.25) Uniform(L, U), and [r−1] [r−1] [r−1] T f = Poi(yi,t ; µi,t)Poi(yj,t ; µj,t)Poi ξi,j,t ; exp Di,jγt × (2.26) Uniform(L∗,U ∗). [r−1] [r−1] [r−1] ∗ ∗[r] ∗ ∗[r] ∗[r] Here L = L(ξi,j,t ), U = U(Yi,t ,Yj,t ). Let L = L(ξi,j,t), and U = U(Yi,t ,Yj,t ). The tuning parameter M should be adjusted so that the acceptance rates of the T Metropolis-Hasting fall in the range 0.1−0.4. The initial values of Y t = (Y1,t..., Ym,t) T can be set to be observations Xt = (X1,t..., Xm,t) for any t = 1, ..., T . And the ini- P tial values of ξt are set to be zeros, such that all the constrains Xi,t = Yi,t + j6=i ξi,j,t are satisfied for i, j = 1, ..., m and t = 1, ..., T . 2.4 Simulation Studies Simulation study was performed to investigate the estimation accuracy and proper- ties of the above described MCMC algorithm. Samples were simulated from the joint 29 posterior distribution under different scenarios. The 2.5th percentile and the 97.5th percentile were used to construct the 95% credible interval to evaluate the model. Here we consider the model without joinpoints. The results for the model with a single covariate on the main effect parameter and a single covariate on the covariance effect parameter (P = 1,Q = 1) are given in Table 2.1. All simulations used settings 2 2 T = 12 and σp and τq were assumed to be known. For simplicity, two cases were 2 2 2 2 considered, σp = τq = 0.1, and σp = τq = 0.03 with number of m locations, where m = 20 and m = 40 were used. For the evolution equations, both equation (2.14) and (2.18) were considered. Note that, in the ARIMA(1,1,0) model, κ and ρ were assumed to be a known positive value between 0.2 and 0.3. The parameter vectors in this model were βp = (β1,1, β1,2, ..., β1,T ), and γq = (γ1,1, γ1,2, ..., γ1,T ), with tuning parameter M = 1. Table 2.1: Simulation results for parameter β and γ in P = 1, Q = 1 settings m=20 m=40 σ2=0.1 σ2=0.03 σ2 =0.1 σ2=0.03 ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) coverage for β 0.95 0.94 0.95 0.94 0.88 0.85 0.94 0.92 coverage for γ 0.93 0.91 0.96 0.94 0.87 0.83 0.94 0.97 sd of β 0.22 0.25 0.21 0.32 0.24 0.40 0.17 0.34 sd of γ 0.46 0.35 0.38 0.48 0.39 0.27 0.20 0.23 30 Table 2.2: Simulation results for parameter β and γ in P = 2,Q = 1 settings m=20 m=40 σ2=0.1 σ2=0.03 σ2 =0.1 σ2=0.03 ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) ARIMA(1, 1, 0) AR(1) coverage for β 0.82 0.83 0.86 0.88 0.86 0.78 0.86 0.87 coverage for γ 0.93 0.88 0.95 0.94 0.92 0.89 0.93 0.95 sd of β 0.63 0.80 0.43 0.27 0.32 0.37 0.18 0.18 sd of γ 0.49 0.47 0.23 0.16 0.23 0.34 0.20 0.19 in Table 2.1 the credible interval coverage for β is adequate in m = 20 case, but it gets worse for m = 40 especially when σ2 takes larger value. The coverage rate for γ appears very similarly to the case of β. We then look at differences between σ2 = 0.1 and σ2 = 0.03. One would expect standard deviation for the parameters in σ2 = 0.03 case to be smaller than in σ2 = 0.1 case. However, the standard error for β and γ in σ2 = 0.03, m = 20 AR(1) are somewhat larger than the standard deviation in σ2 = 0.1 case. This may reflect some other unknown properties of the algorithm. Turning to comparisons between ARIMA(1, 1, 0) and AR(1), there is no substantial differences in coverage. The standard deviation for β in ARIMA(1, 1, 0) are smaller than the standard deviation in AR(1) regardless m and σ2. But standard deviation for γ does not show the same pattern. It seems that m is the driving factor that affects the standard error of γ, where γ shows smaller standard deviation in m = 40 than in m = 20. In order to further validate the algorithm, the results for the model with two co- variates are given in Table 2.2. The other settings are identical to Table 2.1. The 31 parameter vectors in the second model are βp = (βp,1, βp,2, ..., βp,T ), p ∈ {1, 2} and γ1 = (γq,1, γq,2, ..., γq,T ), m = 1. Table 2.2 shows that β gives notably low coverage, reduced by up to 13% for β in m = 20, σ2 = 0.1, in ARIMA(1, 1, 0) and by up to 11% for β in m = 20, σ2 = 0.1, in AR(1), compared with univariate model. It is worst for β in m = 40 AR(1), being down to 78%. Despite the low coverage in β, coverage for γ was adequate (0.88−0.95) in all settings. It comes to identical conclusion to univariate model, while comparing the overall cov- erage between σ2 = 0.1 and σ2 = 0.03 that, larger σ2 shows decreased coverage for both γ and β. The differences in standard deviations between case σ2 = 0.1 and case σ2 = 0.03 appear to satisfy our expectation that, standard deviations for both β and γ appear to be smaller in case σ2 = 0.03 than σ2 = 0.1. 2.5 Analysis of Pertussis Data In this section we consider a pertussis data example to demonstrate the performance of our DMVP model and compare it with a standard Poisson regression. The trans- mission and mortality associated with the highly infectious children disease pertussis decreased substantially after the universal introduction of vaccination in the 1940s − 1960s [34]. However, the incidence has been increasing since the 2000s in different countries, including the developed countries with high vaccine coverage [35, 34]. This 32 resurgence of pertussis has been a public health concern since the recent epidemiolog- ical features of pertussis outbreaks among children are not very well understood [34]. Studies have shown that the resurgence might be the consequence of failures incurrent vaccine effectiveness due to pathogen adaptation [36]. Understanding the association of pertussis transmission and vaccine effectiveness is important for providing valu- able insights into the mechanism of spread, and evaluating optimal intervention and control. The assessment of vaccine effectiveness involves numerous laboratory work and relies on mathematical models as well. 2.5.1 Background Pertussis also known as whooping cough, is a highly contagious respiratory disease that is mostly prevalent among infant and children [37]. Bordetella pertussis is the causative agent of pertussis. The upper respiratory system is attached by the bacteria that releases toxins causing damage on the cilia of respiratory system [38]. The circulation of pertussis is usually through sneezing and coughing or while staying close to one another. Incubation period for pertussis commonly ranges from 7 - 10 days and it usually lasts for 6 weeks. There are 3 stages during the course of pertussis infection: 1) catarrhal state typically last for one to two weeks and is the most contagious state[39, 40]; 2) paroxysmal state that usually last 1-6 weeks and typically terminate with coughing; 3) convalescent state that usually last 7-10 days and might followed by subsequent respiratory infections for months. Pertussis may lead to life threatening complications in babies and young children, especially for 33 those have not received all recommended pertussis vaccines [41], such as pneumonia, convulsions, apnea, and encephalopathy [42]. Adults and teens can get complications from pertussis as well, but less serious compared to infants [43]. Pertussis vaccine can help project against pertussis infection [44, 45]. Two most prevalent types of vaccines are whole-cell vaccines and acellular vaccines [44, 45]. While getting vaccines shots is the most effective way to prevent this disease, it is known that no vaccine is guaranteed to be totally effective [46, 47]. Though it is still possible to get sick for a vaccinated person, the infection is not as bad as in the unvaccinated [38]. Pertussis Vaccine Development History In the United States, DTaP is the recommended vaccine for pertussis especially for babies and children. DTaP is a combination vaccine that helps protect against not only pertussis but also against diphtheria and tetanus. DTaP has been available on the market since 2005 for prevention and treatment [48]. The development of pertussis vaccine dates back in early 1900s, when pertussis was a devastating disease and people started thinking about developing the vaccine [49]. In 1930s, pertussis vaccines were first reported by Madsen and were used for prevention and treatment [50]. During that period, extensive attempts have been made to develop effective whole-cell vaccines. The universal use of these whole-cell vaccines tremendously decreased the incidence rate of pertussis in mid-1950s [51]. However, because of serious adverse to whole cell pertussis vaccine (DTwP), acellular pertussis vaccine (DTaP) also went into development[51]. Between 1962 and 1977 an extracted vaccine 34 Table 2.3: History of pertussis vaccines 1906 B. pertussis isolated 1923−1924 Several vaccine trials by T. Madsen; some protection 1933−1942 Many candidate vaccines 1944 Pertussis vaccine endorsed by AAP 1947 Mouse protection test developed 1981 Acellular vaccines used in Japan (DTaP) was prevalent in the United States and was thought to be less reactogenic although less effective than whole-cell vaccines[51]. During the 1950s-2000, extensive research with B. pertussis was investigated in molecular microbiologists laboratories [52]. Various types of B. pertussis toxin (PT) proteins were identified and were examined in mice. In early 1980s a NIH-supported clinical trial was carried out in Sweden[53] to evaluate the efficacy of a single PT vaccine and a combined vaccine PT/FHA(filamentous hemagglutinin). Both vaccines demonstrated good efficacy of PT vaccine 54% and PT/FHA efficacy of 69%. In 1990s, a large cohort trial involving 82,892 infants was conducted in Sweden to access the efficacy and relative risk of PT/FHA and 3 other types multi-component vaccines of PT vaccines [54, 55]. It was shown that the five-component vaccine was considerably better than other vaccines. After combined with HiB and polio, this five-component vaccine has become routinely used in the United States. Table 2.3 and 2.4 describe the history of the pertussis vaccines development. 35 Table 2.4: History of acellular pertussis vaccines 1930s B. pertussis isolated 1923−1924 Many DTP vaccine preparations studied including Bextracted vaccines 1962−1977 An extracted vaccine (TriSolgen) available in the USA 1978−1981 DTaP vaccines developed and evaluated in Japan 1981 DTaP vaccines used routinely in Japan; starting at 2 years of age 1980s Many DTaP vaccines developed in North America and Europe 1990s DTaP vaccine efficacy trials in Europe and Africa 1995−1997 DTaP vaccines replaced DTwP vaccines in many countries 2006 Tdap vaccines for adolescents and adults became available 2.5.2 Modern Resurgence of Pertussis Resources of reported pertussis cases are available on CDC webpage [6]. CDC collect pertussis cases from state health departments through National Notifiable Diseases Surveillance System (NNDSS). The NNDSS surveillance system is helpful for per- tussis surveillance by keeping track of pertussis epidemiological trends in the U.S [6]. Fig. 2.2 illustrates pertussis incidence rate (per 100,000) by five age groups 1) < 1 year; 2) 1−6 years; 3) 7−10 years; 4)11−19 years; and 5) 20+ years from 1990-2016 in the U.S. Infants < 1 year old show the highest rate of reported pertussis. Fig. 2.3 shows yearly number of CDC reported pertussis cases from 1922 to 2016 in the U.S. In the 1940s the pertussis vaccines were introduced universally when cases declined dramatically and maintain low rate until the 1980s. In late 1980s pertussis counts 36 Figure 2.2: Pertussis Incidence by Age Group began increasing by 2005. Data of pertussis incidence by state and year is available on CDC Wide-ranging ONline Data for Epidemiologic Research (CDC WONDER) [56]. CDC WONDER provides a large number of public health related datasets to researchers to support research in public health and to help in decision making and resource allocation. Fig. 2.4 to 2.12 illustrate pertussis incidence rate (per 100,000) by year and state in the U.S from 2008 to 2015. As we observe on Fig. 2.2 2.3 pertussis incidence increased gradually until 2015 when more than 20,000 cases were reported nationwide[5]. 37 Figure 2.3: Pertussis Incidence Cases 2.5.3 Modeling Pertussis Trends Pertussis count was used as the response variable in our model. We incorporated 2 explanatory variables pertussis vaccination coverage and medicaid enrollment rate at state level at each year. Dataset of pertussis vaccination coverage was from CDC Morbidity and Mortality Weekly Report (MMWR), parts of the data are displayed in Table 2.5 [57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73]. Medicaid enrollment rate dataset is available from government resources Centers for Medicare & Medicaid Services (CMS) [74] session of Medicare coverage database [75]. As in regular Poisson regression, it is more relevant to model disease rates instead of 38 Figure 2.4: Pertussis Rate(per 100,000) 2008 counts. Thus, offset on population size is used in our model. In this setting, we allow for the variation of the 2 explanatory variables over time following AR(1). In addition, we assume that the correlation only exists between the neighboring states. We adopt our DMVP model to fit the data and evaluate model fitting on 9 states: Wisconsin, Illinois, Michigan, Indiana, Ohio, Pennsylvania, New York, Vermont, New Hampshire from 2000 − 2015 illustrated in Fig. 2.13. Therefore, we have m = 9 and T = 16. 2.5.4 Analysis Details T Recall that the model assumes that the outcome variables X = (X1,1, ..., X9,16) follow the decomposition scheme to construct correlation structure: 39 Figure 2.5: Pertussis Rate(per 100,000) 2009 observation equations: X Xi,t = Yi,t + ξi,j,t j6=i Yi,t ∼ Poi(µi,t) log(µi,t) = log(Population)i,t + β0,t + β1,t×(vaccination)i,t + β2,t × (medicaid enrollment)i,t ξi,j,t ∼ Poi(θi,j,t) log(θi,j,t) = γ0,i,j+γ1,i,j × average(population/100000)i,j,t, (2.27) system equations: β = β + w , w ∼ N(0, σ2 ), (2.28) i,t i,t−1 β,t β,t βi 40 Figure 2.6: Pertussis Rate(per 100,000) 2010 Note that the number of parameters in β1 and β2 is equivalent to the total time points T = 16. The two vectors of parameters β1 and β2 are related to the observed counts in all 9 states at each time. We would like to obtain a model that fits the observed data well without overfitting. In order to judge if the DMVP overfits or not, we assess the performance of the model with each of the data removed from the training set and compare the performance to a simple Poisson regression model, where the outcome variables are assumed to follow independent Poisson distributions. Two additional time related covariates are 41 Figure 2.7: Pertussis Rate(per 100,000) 2011 added to the simple Poisson regression model to capture the trend over time. Xi,t∼Poi(µi,t) Xi,ti ⊥ Xj,tj for i 6= j log(µi,t) =log(Population)i,t + β0 + β1 × (vaccination)i,t+ (2.29) 2 β2 × (medicaid enrollment)i,t + t + t To estimate bias and variance of the DMVP model predictions, at each location and at every time point a data point is removed and its value imputed from the model trained on the remaining datapoints. The model is then used to predict the removed locations values (this may be considered as a version of the cross-validation procedure). The results are presented graphically in Fig. 2.14. The dynamic MVP model fit to the pertussis data using time varying coefficients is shown in the top panel of Fig. 2.14. For comparison, a simplified Poisson fit to the data with time as 42 Figure 2.8: Pertussis Rate(per 100,000) 2012 quadratic term is shown at the bottom of Fig. 2.14. Not surprisingly, it appears that the DMVP fit is in general better, in terms of capturing the dynamic variation of the pertussis trend over time, as the simple Poisson regression model is too conserva- tive and averages the incidence rates over all locations disregarding the information from incidence rates of neighboring states. However, we note that the mean square error (MSE) of DMVP model is larger than the simple Poisson regression. This is because DMVP model results in unstable time-varying estimates, which are sensitive to the extreme values in the data. Once one or more of these outliers are removed from the training set, the estimate values improve considerably. For instance, we observe two peaks in year 2004 and 2012 in several among the 9 states. In 2012, the states of Pennsylvania, Vermont, New York, and Wisconsin experienced sudden 43 Figure 2.9: Pertussis Rate(per 100,000) 2013 increase in pertussis rate that resolved after 2013. Thus, the estimates in year 2012 are driven towards a higher incidence rate by these 4 states who experienced the peak, and especially are dominated by the ones with large state population sizes such as Wisconsin and Vermont. On the other hand, the model over-estimates the rate in some of the states that didn’t experience the sudden increase such as New Hampshire. This explains the big discrepancies in 2012 between predicted values and true observations for state of Vermont, New Hampshire, and Wisconsin in Fig. 2.14. Similarly, in 2004 it seems that Illinois and Indiana were the only 2 states showing a high rate (although lower than in 2012) and for both states the model predictions were biased downward as it considered these as local fluctuations. This is perhaps to be expected given a relatively small size of the training dataset. 44 Figure 2.10: Pertussis Rate(per 100,000) 2014 In agreement with the cross-validation predictions, DMVP model also did not find any significant jointpoints, although the potential joinpoints were initially pre-specified at year 2003, 2006, 2009, and 2014. 2.5.5 Conclusion The choice of an appropriate model plays an important role in the evaluation of diseases transmission and prevention [76, 77, 78, 79, 80]. There has been intensive discussion on the advantages of a conventional (static) models vs dynamic ones used for prediction purposes in public health and epidemiology. Static model assume that the probability of disease exposure and the effect of intervention do not change over time [76]. Whereas these assumptions my be realistic for non-transmissible 45 Figure 2.11: Pertussis Rate(per 100,000) 2015 diseases, for infectious diseases they are typically not valid and hence the need for dynamic models which are essential to study the rate of infection dynamics and it temporal/spatial trends in order to help control transmission [81, 82, 83, 84, 85]. For the pertussis dataset, the DMVP fits better than naive Poisson regression, largely due to its dynamic character and ability to borrow temporal/spatial information. In the next chapter, via the notion of GOF analysis, we will more formally examine the issue of models comparison agains data. 46 Figure 2.12: Pertussis Rate(per 100,000) 2016 Figure 2.13: 9 States 47 Table 2.5: Pertussis Vaccination Coverage (%) 2008 2009 2010 2011 2012 2013 2013 2014 2015 Illinois 74.8 72.8 75.1 71.8 66.5 66.8 68.3 70.8 Indiana 75.5 67.3 71.6 70.1 61.4 68.5 66.3 74.7 Pennsylvania 77.7 69.4 77.1 73 68.3 75.5 78.6 72.8 Vermont 64.5 59.9 69.7 73.4 63.2 66.9 71.8 75.6 Michigan 74.5 76.5 81.3 71.8 70.5 70.0 65.0 67.7 New Hampshire 81.0 73.3 81 72.6 80.1 74.9 80.4 74.1 New York 73.3 67.2 64.1 65.1 63.7 72.2 70.7 71.9 Ohio 81.8 72.4 76.2 76.4 66.8 61.7 68.1 68.3 Wisconsin 79.6 75.9 82.7 79.2 75.2 72.8 70.9 68.8 48 Figure 2.14: DMVP and Simplified Poisson Model Fitting 49 Chapter 3: The Goodness of Fit (GOF) in Generalized Linear Model (GLM) 3.1 Summary of Previous Research on GOF T The question of measuring the discrepancy between the fitted values Yb = (ˆy1, yˆ2, ..., yˆn) T and the observations Y = (y1, y2, ..., yn) arises once model is fitted. The concept of a goodness of fit of a model reflects typically some quantification of a distance between the observations and the model prediction. Two extreme reference models are often of interest in the context of fit analysis. The first one is the saturated model in which the number of estimated parameters is equal to the number of data points. The saturated model realizes the minimal bias for a given dataset. The other is the simplest constant model, also referred as null model, in which the grand mean µ is used for predicting the values for all data points. The null model realizes the minimal variance for a given dataset. Of course, in practice, neither the null model nor the saturated model are of interest. Instead, an intermediate model with P covariate patterns is considered to compare with the null and saturated models. 50 Covariate patterns refer to the distinct values of the subsets of explanatory variables [86]. Identifying and enumerating all covariate pattern is a strategy for evaluating model fitting in a number of GOF methods such as the Hosmer-Lemeshow method [86]. However, this strategy has strong limitations in many real datasets, for in- stance when having continuous covariates that need to be grouped. The results of GOF analysis may then depend of the way of grouping/aggregating data points, ad- ditionally large number of patterns may complicate the asymptotic analysis of GOF. To illustrate, let n denote the total number of data points and J be the number of covariate patterns. J < n, if some of the observations share the same covariance pat- tern. The asymptotic distributions for various GOF statistics are typically obtained when the total number of observations n → ∞ while the number of covariate pat- terns J is fixed. However, if continuos variables are contained in the model, then we may have J ≈ n and the covariates grouping may not be particularly advantageous. Goodness-of-fit testing for GLMs, especially for discrete/binary outcomes is of great interest to applied statisticians [87]. We will introduce several currently used GOF test statistics in the next section. Then, we will also propose a new GOF test statics (denoted below by Cn) and show its asymptotic properties. In order to evaluate the performance of Cn we will use simulated numerical examples, but derived from practical applications and datasets in several different areas of social science and public health. Due to importance of the binary data in public health application, 51 we separately consider GOF problems for binary and other (non-binary) discrete outcomes. 3.1.1 GOF Tests for Binary Outcomes Hosmer-Lemeshow Statistic In 1980, Hosmer and Lemeshow [88] proposed a grouping-based method to measure the goodness of fit for the binary response model. Specifically, the data are grouped according to the estimated probability. The observed and expected frequency are calculated for each group to obtain the test statistic G " 2 2 # X (O1g − Ebig) (O0g − Eb0g) HL = + g=1 Eb1g Eb0g th Pcg where g denotes the g group, g = 1, 2, 3, ..., G. O1g = j=1 Yi, is the observed th number of events (Yi = 1) in the g group, where cg is the number of covariate th Pcg patterns in the g group. Eb1g = j=1 Njpˆj is the expected number of events in the th th g group, where Ng is the number of observations in g covariate pattern, andp ˆj is th Pcg the estimated probability for j covariate pattern. Similarly, O0g = j=1(Nj −Yj) is th Pcg the observed number of non-events (Yi = 0) in the g group, and Eb0g = j=1 Nj(1− th pˆj) is the expected number of non-events in the g group. Since G = 10 and G = 20 are commonly suggested in practice, we use these two settings to perform HL tests in Section 3.3. With a bit of algebra, it can be shown that G 0 2 X (O1g − ngp¯g) HL = n0 p¯ (1 − p¯ ) g=1 g g g 52 0 th where ng is the number of observations in the g group,p ¯g is the average estimated probability in the gth group. When the number of groups G equals the sample size n, the formula for HL can be further rewritten as n 2 X (Yi − pˆi) HL = pˆ (1 − pˆ ) i=1 i i which suggests the formula for another goodness of fit measure: the sum of squared standardized residuals. Stukel Statistic Stukel et al.[89] proposed a convenient GOF model based on adding two additional covariates in logistic regression to test for the presence of quadratic terms. The idea also applies to general GLMs. The two additional covariates proposed in Stukel 2 2 T ˆ model are za = ζi I(ζi ≥ 0) and zb = ζi I(ζi < 0), where ζi = Zi β. By adding za T ˆ ˆ ˆ and zb into the model, the augmented model is ζi = log(µi) = Zi β + βaza + βbzb. With this model, the Wald test or the likelihood ratio test is performed to test H0 : βa = βb = 0. Unweighted Sum of Squares Statistic When some of the expected cell frequencies in the 2 × n contingency table are too small, the weighted chi-squared test for proportions cannot be used. In this case, Copas [90] modified the chi-squared test as n X 2 USS = (Yi − pˆi) , i=1 53 wherep ˆi is the estimated probability, n is the sample size. Under the null hypothesis, this statistic will have an asymptotic normal distribution when standardized by its mean and the square root of its variance. Information Matrix Statistic The information (IM) test was firstly introduced by White et al. [91] to examine general model misspecification with the help of information matrix. IM statistic compares two estimates of the variance covariance matrix of the regression coefficients β, one based on square of the first derivatives of the log-likelihood function and the other based on the second derivatives. If the model is correctly specified, the two estimates should agree. n 2 X ∂ logf(Yi; β) A (β) = n−1 n ∂β ∂β i=1 j k n (3.1) X ∂logf(Yi; β) ∂logf(Yi; β) B (β) = n−1 . n ∂β ∂β i=1 j k When the above are invertible, we may define Σn(β) −1 −1 Σn(β) = An(β) Bn(β)An(β) . and Σ(β) = limn Σn(β). Theorems 3.1 and 3.2 in White [91] states that, √ ˆ n(βn − β0) → N(0, Σ(β0)) as n → ∞. However, it has been noted that the IM statistics is highly sensitive to model mis- specification due to the unreliable gradient in the covariance estimation [92]. Al- though Newey[93] has proposed another calculation procedure for the gradient, it 54 relies on the use of higher moments estimates that may be sometimes also problem- atic, as was argued by Davidson and Mackinnon [94]. Davidson and Mackinnon [94] suggested to use the parametric form of the covariance matrix and replace unknown parameters by their MLEs. However their covariance matrix estimates are derived for binary model only, and the implementation for other GLMs was not evaluated. In this work, we use the covariance estimate of the IM statistic for logistic regression derived by Davidson and Mackinnon[94] who simplified the IM statistics for binary model with logit link to be n p X X 2 IM = (Yi − pˆi)(1 − 2ˆpi)Zij i=1 j=0 th th where Zij is the j predictor for i observation. It has been shown by White [91] that IM has a approximate chi-square distribution with p + 1 degree of freedom under the null hypothesis of the correct model. Sum of Squared Residuals As we mentioned previously, the sum of squared standardized residuals is also a goodness of fit measure in binary response model n 2 n X (Yi − pˆi) X T = = W . n pˆ (1 − pˆ ) i i=1 i i i=1 This statistic was proved to be asymptotically following the standard normal dis- tribution when properly standardized [95]. However, the drawback of Tn statistic is that the variances of its individual components Wi depend heavily on pi in such 55 way that extreme numerical values of pi could produce highly unbalanced variance in Tn for different n. One intuitive way to address this issue is to assign a variance- stabilizing weight to Wi, which motivates our proposed test statistic Cn discussed in the next section. 3.1.2 GOF Tests for Discrete Outcomes In recent years relatively little work has appeared in the literature on GOF effort for more general GLMs. We summarize some of the most important ones below with some of them already discussed in previous section. Pearson X2 Statistic Pearson’s X2 statistic is one of the most widely used GOF statistics, being first proposed by Karl Pearson in 1900 [96] to evaluate GOF for multinomial data. The test statistics has been used to test for independence, but then extended to test for model fitting [97]. The statistic is J ˆ 2 X (Yi − µ(θi)) X2 = , V (ˆµ ) i=1 i ˆ where µ(θi) and V (ˆµi) are the estimated mean and variance for the ith observation ˆ respectively and for convenience we writeµ ˆi = µ(θi). For some distributions in exponential family such as Poisson and binomial distribution, X2 is the original Pearson X2 statistic, while for the normal distribution, X2 is the same as the residual sum of squares. Under the null hypothesis when the fitted model is correct, X2 is 56 approximately distributed as χ2 with the same number of degrees of freedom as the residuals. The results for calculating large sample approximations for the mean and variance of the Pearson chi-square statistic for binary data was summarized by McCullagh et al. [20]. Osius and Rojek [98] tackled this problem and showed that the asymptotic distribution of the Pearson-based test statistic is Normal (0, 1). Equivalent results is shown by Windmeijer [95]. Deviance Another commonly used GOF test statistic is the deviance, which is a log-likelihood based statistic. Let `(βˆ; y) be the log-likelihood achieved by maximizing over β given the dispersion parameter φ. Deviance is proportional to the difference between the maximum log likelihood of the saturated model and the model based on the T covariates Z = (z1, z2, ..., zp) . The general formula for deviance is, D(y;µ ˆ)/φ = −2 log `(y|βˆ) − log `(y|β˜) . (3.2) Here βˆ and β˜ are the MLEs of the saturated model and the model with p regression coefficients, respectively. The deviance statistics for some commonly used GLM distributions are: • Normal n X 2 (yi − µˆi) . i=1 • Poisson n X 2 2 {yilog(yi/µˆi) − (yi − µˆi)} . i=1 57 • Binomial J X n Ni − yi o 2 y log(y /µˆ ) − (N − y )log . i i i i i N − µˆ i=1 i i • Gamma n X n yi − µˆi o 2 − log(y /µˆ ) + . i i µˆ i=1 i Both Deviance and Pearson X2 statistic have asymptotic χ2 distribution under the null hypothesis with the large values of these statistics indicating lack of fit. Williams et al. [99] exploited the one step approximation for the changes in the deviance of a GLM when a single data point is deleted from the sample. Their approximation suggests a particular set of residuals which can be used to calculate measures of the influence of observations. Inferences can be drawn from the fitted model using likelihood ratio statistics. However, it is generally acknowledged that both Pearson’s X2 Statistic and Deviance have several drawbacks. For instance, both statistics no longer have a chi-square distribution when the number of categories (sample covariate pattern) increases with sample size [95] and consequently the traditional GOF p-values for Pearson statistic and Deviance are no longer valid. Information Matrix (White) As discussed in Section 3.1.1, the efficient covariance estimator of the variance of IM statistics was only derived for binary distributions. The use of the IM statistics in this work for discrete GLMs is based on the original article White et al. [91]. Here, 58 we find below that the quality of the estimate of the covariance matrix can improve with large sample size. 3.2 Proposed Test Statistic - Standardized Residuals In this section we provide the definition and derive basic properties of our proposed GOF statistic. We assume that the outcome random variable Y follows natural exponential family: (yθ − b(θ)) f (y; θ, φ) = exp{ }c(y; φ), (3.3) Y α(φ) and ( 0 E(Y ) = µ = b (θ) Var(Y ) = V (µ) = b00 (θ)α(φ). Recall that we define the link function as g(µ) = θ = ZT β, where Z the design matrix with p covariates. Recall that in the canonical setting µ can be expressed −1 ˆ as the inverse of the link function, µ = g (θ). The MLE of β0 is denoted by βn, where β0 is the p-vector of coefficients under the null hypothesis. The proposed test statistic for the goodness of fit is the sum of weighted residuals [100] n ˆ X Yi − µ(θi) Cn(βbn) = p , i = 1, ..., n. i=1 V (ˆµi) 0 T 1 Note that for certain distribution functions where ψ (Zi β) ∝ √ , the sum of V (µi) standardized residuals is a degenerate statistic due to the particular form of the score equations in such case. This is the case for most continuos distributions such 59 as normal and gamma. We will present an example to illustrate this degeneracy property later in this section. (i) ∂V (µi) has continuos first derivative with respect to β in the neighborhood of ∂β0 ∂V (µi) β , such that ∂β0 < ∞. 0 V (µi) p (ii) β0 is contained in the parameter space B, an open bounded subset of the R . ∂µ(θi) −1 Pn ∂µ(θi) ∂µ(θi) (iii) are uniformly bounded in i and limn→∞n i=1 T is a finite ∂β0 ∂β0 ∂β0 nonsingular matrix. The empirical distribution of ∂µ(θi) converges to a non- ∂β0 degenerate distribution. (iv) Yi are independent random variables. (4) b(4)(θ) (v) b(θ) has continuos fourth derivative b (θ) and φ < ∞ for θ ∈ Θ ⊂ R. Under these assumptions, it can be proved that the asymptotic distribution of max- imum likelihood estimator βbn of β0 [101, 102] satisfies 1 √ (βb − β ) → N(0, Ω−1) as n → ∞. n n 0 Here n 2 1 X ∂log f(Yi) Ω = lim n→∞ n T i=1 ∂β∂β (3.4) = Rn(β) − Mn(β) , 60 where n X 00 T T Rn(β) = (yi − µ(θi))ψ (Zi β)Zi Zi, i=1 and n X 2 00 T T Mn(β) = (yi − µ(θi)) b (ψ(Zi β))Zi Zi. i=1 Theorem 1. Under the assumptions 1–5, it can be shown that as n → ∞ − 1 n 2 Cn(βb ) d n −→ N(0, 1), σn where →d denotes convergence in distribution and 2 T −1 σn = 1 − νn Ω νn, with n 1 X 1 ∂µ(θi) ν = . n n p ∂β i=1 V (µi) 0 Proof. The following can be obtained by Taylor’s expansion: ∂log`(β) ∂log`(β) ∂2log`(β) | = |β + |β (βbn − β0) + op(1). (3.5) ∂β βbn ∂β 0 ∂β∂βT 0 The log-likelihood function is n X hyiθi − b(θi) i log`(θ) = + log c(y ; φ ) . φ i i i=1 i The left hand side of equation (3.5) is 0, because βbn is the maximizer of log`(θ). Rearranging h∂2log`(θ) i−1 ∂log`(θ) βb − β = − |β |β n 0 ∂β∂βT 0 ∂β 0 (3.6) ∂log`(β) = Ω−1 | + o (1), ∂β β0 p 61 where op(1) is a random variable with probability limit 0. Local expansion of Cn(βbn) around β0 gives 1 1 1 ∂Cn(β) √ √ Cn(βb ) = √ Cn(β ) + |β n(βb − β ) + op(1). n n n 0 n ∂βT 0 n 0 1 ∂Cn(β) | can be broken down into two parts, n ∂βT β0 n 1 ∂Cn(β) 1 X ∂ Yi − µ(θi) | = n T β0 n ∂β p ∂β i=1 0 V (µi) n ∂µ(θi) p ∂ p 1 X − ∂β V (µi) − (Yi − µ(θi)) ∂β V (µi) = 0 0 n V (µ ) i=1 i ∂µ(θi) p 1 ∂V (µi) n √ − ∂β V (µi) − (Yi − µ(θi)) ∂β 1 X 0 2 V (µi) 0 = n V (µ ) i=1 i (3.7) ∂µ(θi) p 1 ∂V (µi) ∂µ(θi) n √ − ∂β V (µi) − (Yi − µ(θi)) ∂µ ∂β 1 X 0 2 V (µi) i 0 = n V (µ ) i=1 i n ∂V (µi) (Yi − µ(θi)) 1 X h ∂µi 1 i∂µ(θi) = − 3 + p n 2 ∂β i=1 2(V (µi)) V (µi) 0 = U n − νn. Here n ∂V (µi) (Yi − µ(θi)) 1 X ∂µi ∂µ(θi) U n = − 3 (3.8) n 2 ∂β i=1 2(V (µi)) 0 and n 1 X 1 ∂µ(θi) ν = . (3.9) n n p ∂β i=1 V (µi) 0 62 P Under conditions 1-5, it can be shown that U n −→ 0 as n → ∞, because E(U n) = 0 and n ∂V (µi) 2 1 X V (µi) ∂µ ∂µ(θi) 2 Var(U ) = i n n2 3 ∂β i=1 4 V (µi) 0 n 1 X V (µi) ∂V (µi) 2 = (3.10) n2 3 ∂β i=1 4 V (µi) 0 n 1 X 1 ∂V (µi) 2 = . n2 2 ∂β i=1 4 V (µi) 0 Thus, Var(U n) → 0 as n → ∞. Note that νn is a p-dimensional vector n 1 X 1 ∂µ(θi) ν = , β = (b , b , ..., b )T . n n p ∂β 0 0 1 p i=1 V (µi) 0 According to the Jensen inequality ([102] Chapter 1) the sequence νn is bounded. Indeed, note n 1 X 1 ∂µ(θi) ||ν ||2 = || ||2 n n p ∂β i=1 V (µi) 0 n n 2 2 1 X 1 ∂µ(θi) 1 X 1 ∂µ(θi) = + ... + n p ∂b n p ∂b i=1 V (µi) 0 i=1 V (µi) p n n 1 X 1 ∂µ(θi) 2 1 X 1 ∂µ(θi) 2 + ... + (3.11) 6 n p ∂b n p ∂b i=1 V (µi) 0 i=1 V (µi) p n 1 X 1 ∂µ(θi) 2 1 ∂µ(θi) 2 = + ... + n p ∂b p ∂b i=1 V (µi) 0 V (µi) p n 1 X 1 ∂µ(θi) ∂µ(θi) = || ||2, n p ∂β T i=1 V (µi) 0 ∂β0 which converges due to assumptions 1-5. Thus, from equation 3.6 it follows that, T T −1 ∂log` ν (βb − β ) = ν Ω |β + op(1) (3.12) n n 0 n ∂β 0 63 √1 and therefore n Cn(βbn) has the same limiting distribution as 1 1 ∂log` √ C (β ) − √ νT Ω−1 | . (3.13) n n 0 n n ∂β β0 By differentiating the likelihood (cf. [102] (4.59)) we see that n ∂log`(θ) X 0 Zi = (Y − µ(θ ))ψ (ZT β) . (3.14) ∂β i i i φ i=1 i √1 Thus, the asymptotic distribution of n Cn(βbn) is the same as n 1 1 X 0 Zi √ C (β ) − √ νT Ω−1 (Y − µ(θ ))ψ (ZT β) n n 0 n n i i i φ i=1 i n n 1 X Yi − µ(θi) 1 X 0 Zi = √ − √ νT Ω−1 (Y − µ(θ ))ψ (ZT β) (3.15) n p n n i i i φ i=1 V (µi) i=1 i n 1 X 1 0 Zi = √ Y − µ(θ ) − νT Ω−1ψ (ZT β) . n i i p n i φ i=1 V (µi) i Now define 1 T −1 0 T Zi Dni = Yi − µ(θi) p − νn Ω ψ (Zi β) . V (µi) φi Note that, Dn1, ..., Dnn are independently distributed and E(Dni) = 0 as well as 2 1 T −1 0 T Zi V ar(Dni) = V (µi) p − νn Ω ψ (Zi β) V (µi) φi (3.16) 2 = σni. 64 th The 4 moment of Dni is 4 4 4 1 T −1 0 T Zi E(Dni) = E Yi − µ(θi) p − νn Ω ψ (Zi β) V (µ(θi)) φi 4 3 2 2 3 4 4 = E Yi − 4Yi µ(θi) + 6Yi µ(θi) − 4Yiµ(θi) + µ(θi) ai , " # 1 0 Z T −1 T i (3.17) where we denoted ai = p − νn Ω ψ (Zi β) V (µ(θi)) φi 4 3 2 2 3 4 4 = EYi − 4EYi µ(θi) + 6EYi µ(θi) − 4EYiµ(θi) + µ(θi) ai 4 3 2 2 4 4 = EYi − 4EYi µ(θi) + 6EYi µ(θi) − 3µ(θi) ai . th Next, we need to evaluate that the 4 moment of Dni is bounded. The moment generating function (m.g.f) for GLMs is derived as follows Z ∞ nyθ − b(θ)o E(ety) = etyexp c(y; φ)dy −∞ φ Z ∞ ny(tφ + θ) − b(θ)o = exp c(y; φ)dy −∞ φ Z ∞ ny(tφ + θ) − b(θ + tφ) + b(θ + tφ) − b(θ)o = exp c(y; φ)dy (3.18) −∞ φ nb(θ + tφ) − b(θ)o = exp φ = MY (t). In terms of m.g.f. the fourth moments of Y is expressed as ∂4M E(Y 4) = M (4)(0) = Y (0) (3.19) Y ∂t4 that is ∂4M b00 (θ) b0 (θ) b000 (θ) b0 (θ) b00 (θ) b00 (θ) b000 (θ) Y (0) = 2 + 2 + 4 2 + 3 ∂t4 φ φ φ φ φ φ φ b0 (θ) b00 (θ) b0 (θ) b(4)(θ) b0 (θ) b0 (θ) b00 (θ) + 7 2 + 4 2 + 4 + 3 3 . φ φ φ φ φ φ φ (3.20) 65 4 From above we see that mni = E|Dni| is bounded under the assumptions 1-5. It now follows that Pn 4 i=1 E|Dni| lim n n→∞ P 2 2 ( i=1 σni) Pn m = lim i=1 ni n→∞ Pn 2 2 i=1 σni Pn max{m } (3.21) lim i=1 ni 6n→∞ Pn 2 2 i=1 min{σni} nm = lim max n→∞ 2 2 (nσmin) =0 Pn Applying Lyapunov’s Central Limit Theorem (see [102] Chapter 1) to i=1 Dni the √1 asymptotic distribution of n Cn(βbn) can now be obtained, since √ Yi−µ(θi) T −1 0 T V (µi) n √ 1 − νn Ω Ziψ (Zi β) φ 1 X V (µi) i d √ √ 1 −→ N(0, 1) n V (µ ) 2 2 i=1 1 Pn T −1 0 T i 1 − ν Ω Ziψ (Z β) n i=1 n i φi and, in view of the definitions of Ω and νn, n p !2 1 X 0 V (µi) 1 − νT Ω−1Z ψ (ZT β) = 1 − νT Ω−1ν + o (1). n n i i φ n n p i=1 i We also note that due to the Toeplitz lemma ([102] Chapter 1) we may replace β in the above variance formula by its MLE βb. 3.3 Numerical Examples Next, we demonstrated that the proposed Cn statistics outperforms the existing methods via numerical examples chosen from different disciplines. We adopt real 66 data examples in simulation studies to investigate the power of proposed test in logistic, Poisson, and negative binomial regressions and compare its performance to that of the existing methods. 3.3.1 Examples for Binary Variables To show the application of the Theorem 1 result and evaluate the power of Cn along with other GOF statistics, we provide several numerical examples under different scenarios. Statistics considered here include Hosmer-Lemeshow [88] (HL) with G = 10 and G = 20, Stukel [89], unweighted sum of squares [90] (USS), Information Matrix [91] (IM) and our new proposed statistic Cn. The tests are all performed at the α level of 0.05. As the entirely artificially simulated covariates might not be very meaningful, we consider the covariates based on values taken from several real datasets. In other words, we only generate the outcome variables Yi for the example datasets below. For each simulation, the number of replication is 10,000. Linear vs. quadratic Covariates used in this section are derived from an epidemiological study conducted among runners in the 2002 Boston Marathon [103]. The goal of this study is to iden- tify potential risks associated with hyponatremia which is defined as a serum sodium concentration level of 135 mmol/l or lower. Details about the selected covariates and the outcome variable are provided in Table 3.1. After excluding participants 67 Table 3.1: Selected variables description in the Boston Marathon data Symbol Codes/Values Variable description y 0 = ≤ 135 mmol/liter, 1 = >135 mmol/liter Serum sodium concentration s 0 = Female, 1 = Male Sex w kg Weight change during race r Hours Race durition u 1 = > 3liter, 0 = ≤ 3liter Fluid consumption with missing values in selected covariates or the outcome variable, we are left with n = 445. Models we used to generate Yi are: exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui) pi = Marathon 1.0 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui) 2 exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.1wi ) pi = 2 Marathon 1.1 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.1wi ) 2 exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.2wi ) pi = 2 Marathon 1.2 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.2wi ) Marathon 1.0 contains no quadratic term, Marathon 1.1 and Marathon 1.2 contain quadratic terms with different coefficients which are to emphasize or deemphasize the effect of the quadratic term. Results comparing the H0 rejection proportion among different tests are showed in Table 3.2. 68 Table 3.2: Proportion H0 rejected at the α = 0.05 using sample size of 445 with 10000 replications in quadratic models using the Boston Marathon data Model Test statistics Proportion H0 rejected Under H0: Marathon 1.0 Cn 0.042 HL(G = 10) 0.059 HL(G = 20) 0.071 USS 0.058 Stukel 0.072 IM 0.051 Under Ha: Marathon 1.1 Cn 0.337 HL(G = 10) 0.224 HL(G = 20) 0.300 USS 0.306 Stukel 0.287 IM 0.234 Under Ha: Marathon 1.2 Cn 0.985 HL(G = 10) 0.847 HL(G = 20) 0.921 USS 0.977 Stukel 0.976 IM 0.964 69 In Table 3.2, we see that under null, nearly all test statistics reject the null at the nominal α level of 0.05 except Stukel and HL(G = 10). Table 3.2 also shows that under Marathon 1.1 which is added a quadratic term with a coefficient of 0.1, rejection proportions of all statistics increase. It is worth mentioning that Cn has the highest power here. Under Marathon 1.2, as the value of the coefficient increases, the power of detecting such departure becomes much greater. Indeed the tests based on Cn, Stukel and IM all have the power exceeding 90%. Linear vs. interaction Additional set of covariates is used to measure power against superficial interaction and comes from the Cleveland dataset which is a subset of a large Heart Disease dataset [104]. Only two variables s and t (see below for descriptions) are chosen to be included in our simulation based on their significant relationship with the response variable. After excluding observations with missing values in any of the three variables, a total of 303 observations is left in our study. Table 3.3 shows a description of these three variables. We used two models to generate Yi: exp(−4 + 1.4si + 0.02ti) pi = Cleveland 2.0 1 + exp(−4 + 1.4si + 0.02ti) exp(−1 + 4.5si + 0.03ti − 0.065si × ti) pi = Cleveland 2.1 1 + exp(−1 + 4.5si + 0.03ti − 0.065si × ti) 70 Table 3.3: Selected variables description in Cleveland data Symbol Codes/Values Variable description y 0 = Absence, 1 = Presence Heart Desease Diagnosis s 0 = Female, 1 = Male Sex t mm Hg, recorded at baseline Resting blood pressure Cleveland 2.0 is the null (hypothesis) model which contains two covariates s and t. Cleveland 2.1 is the alternative model which contains not only s and t, but also an interaction term between the two covariates. Table 3.4 shows the percent of time H0 was rejected by each of the six tests except Hosmer-Lemeshow with group=20 because of the limited number of covariate patterns in this situation. We see in Table 3.4 that, as expected, the proportions of rejecting the null hypothesis are still close to 0.05 when the fitted model is correct. Also, when an interaction term is added to the true model, rejection proportions are greater than those under null. In this setting, while both HL with a group of size 10 and USS attain a poor power, Cn performs noticeably better than the others with a very good power of 65.3%. However, it is unclear whether the increase in power holds when the true model departs further from the fitted model. To find out whether further departure yields higher power, another set of covariates is used. It comes from the National Health and Nutrition Survey and contains 17,030 observations and 16 variables. Two independent variables, m and c, are chosen to 71 Table 3.4: Proportion H0 rejected at the α = 0.05 using sample size of 303 with 10000 replications in interaction models using the Cleveland data Model Test statistics Proportion H0 rejected Under H0: Cleveland 2.0 Cn 0.048 HL(G = 10) 0.033 USS 0.048 Stukel 0.056 IM 0.043 Under Ha: Cleveland 2.1 Cn 0.653 HL(G = 10) 0.263 USS 0.554 Stukel 0.557 IM 0.446 run simulation. They are significantly associated with the dependent variable y simultaneously. We randomly select 373 observations with no missing value in m, c and y. Table 3.5 gives a description of these three variables. We used three models to generate Yi: 72 Table 3.5: Selected variables description in NHANES data Symbol Codes/Values Variable description y 0 = PEPMNK1R ≤140, 1 = PEPMNK1R >140 High Blood Pressure m 0 = Female, 1 = Male Sex c mg/100ml Serum Cholesterol exp(−1.5 + 0.005ci + 1.24mi) pi = NHANES 3.0 1 + exp(−1.5 + 0.005ci + 1.24mi) exp(−0.6 + 0.0007ci + 0.5mi + 0.003ci × mi) pi = NHANES 3.1 1 + exp(−0.6 + 0.0007ci + 0.5mi + 0.003ci × mi) exp(−0.6 + 0.0007ci + 0.5mi + 0.01ci × mi) pi = NHANES 3.2 1 + exp(−0.6 + 0.0007ci + 0.5mi + 0.01ci × mi) NHANES 3.0 is the model under null containing only the linear term, and NHANES 3.1 and NHANES 3.2 are two alternative models designed to assess the power of detecting the omission of a interaction component. The variation in the coefficients of the last two models is used to emphasize and deemphasize the effect of omitting an interaction term. Table 3.6 shows the the percent of time H0 was rejected by each of the six tests using the NHANES data. In Table 3.6, we see that under null, proportions of H0 being rejected among all statistics remain close to the nominal level of 0.05. Except two Hosmer-Lemeshow 73 Table 3.6: Proportion H0 rejected at the α = 0.05 using sample size of 373 with 10000 replications in interaction models using the NHANES data Model Test statistics Proportion H0 rejected Under H0: NHANES 3.0 Cn 0.052 HL(G = 10) 0.038 HL(G = 20) 0.039 USS 0.051 Stukel 0.055 IM 0.050 Under Ha: NHANES 3.1 Cn 0.092 HL(G = 10) 0.049 HL(G = 20) 0.050 USS 0.074 Stukel 0.094 IM 0.065 Under Ha: NHANES 3.2 Cn 0.326 HL(G = 10) 0.095 HL(G = 20) 0.070 USS 0.336 Stukel 0.297 IM 0.226 74 tests, the power of the other tests increases to around 10% when omitting an inter- action term with coefficient of 0.03. As the coefficient increases to 0.01, the power of almost all statistics increases to around 30%. Moreover, only Cn attains high power in both situations. In the subsequent section, we will present data examples applying the Cn to Poisson and negative binomial regression models and compare its performance to GOF tests based on the Pearson, Deviance, Stukel, and IM statistic. We again use real datasets to extract covariates and to generate the outcome variable Yi. Two numerical ex- amples are provided for each of the two models: Poisson and negative binomial. All Monte Carlo estimates are based on 10, 000 repetitions, as before. 3.3.2 Poisson Regression Example 1 Covariates used in the first example are from the Affairs data [105]. The response variable indicates the number of affairs reported by the participant in the past year and modeled as count data. The affair data consists of marriage related variables collected from the study of 601 participants. The selected variables are provided in Table 3.7. Models used to generate independent outcome variables Yi are: Yi ∼ Poisson(µi) 75 Table 3.7: Selected variables description in the Affairs data (n=601) Symbol Codes/Values Variable description y Counts Number of affairs within last year m 0 = Female, 1 = Male Sex a Years Age r Years Years of marrage k Counts Number of kids Table 3.8: Proportion H0 rejected at the α = 0.05 using sample size of 601 with 10000 replications using the Affairs data Model Test statistics Proportion H0 rejected Under H0: Affair 1.0 Cn 0.0506 P earson 0.0495 Deviance 0.7277 Stukel 0.0502 IM 0.0867 log(µi) = 0.2539 + 0.127mi − 0.0316ai + 0.119ri + 0.005ki Affair 1.0 log(µi) = 0.2539 + 0.127mi − 0.0316ai + 0.119ri + 0.005ki + B × mi × ai Affair 1.1 Predictors are gender(m), age(a), years of marriage(r), and number of kids(k). Affair 1.0 is the null model. The Affair 1.1 is an interaction misspecification model, where 76 Figure 3.1: Proportion H0 rejected at the α = 0.05 using sample size of 601 with 10000 replications in interaction models using the Affair data B is the amount of misspecification and takes the values (0.000, 0.005, 0.010, 0.015, 0.020, 0.025, 0.030, 0.035, 0.040). When B = 0, Affair 1.1 is equivalent to the null model, under which the probability of rejecting H0 should remain α = 0.05. The values of the estimated type one error probability for the 5 test statistics are given in Table 3.8 based on 10,000 Monte Carlo simulations. Based on the same number of simulations Figure 3.1 displays the rejection rate of Cn, Pearson, and Deviance statistic. It appears that when B = 0 Deviance doesn’t hold the nominal type one error rate of α (also seen in Table 3.8) and adding misspecification measurements 77 brings little change to the power as the proportion of rejections remain almost con- stant as B increases. We also note that IM has a rejection rate significantly higher than 0.05 in table 3.8. As discussed before, the covariance estimate of IM statistics is only reliable when sample size is sufficiently large. We investigate the sample sized needed for IM in the next section. The statistics Cn, Skutel, and Pearson appear to have close to nominal rejection rate while B = 0. The increase of B leads to higher possibility of rejecting the H0 for all tests, as expected. However, it is observed that the power gain for Cn and Skutel are consistently and substantially larger than Pearson statistic. In this example the validity of the Deviance test seems questionable. 3.3.3 Poisson Regression Example 2 The second example is looking at the question of sample size needed to achieve a prescribed power in the study. Data for this example came from German Credit data [106]. The original dataset was provided by Prof. Hans Hofmann from University of Hamburg and is available at the UCI Repository of Machine Learning Databases [106]. It was used to develop a credit scoring rule to determine the credit risk of a new applicant, based on the customers’ historical financial information. Potential explanatory variables were age (a) in years and credit amount(c) in US dollars. The outcome variable is the duration of the outstanding credit (loan) in months. Table 3.9 provides the description of the variables selected to simulate the outcomes. The models used to generate responses Yi are: 78 log(µi) = −2.79 + 0.097ci − 0.003ai German 2.0 2 log(µi) = −2.79 + 0.097ci − 0.003ai − 0.001ci German 2.1 German 2.0 is the null model. German 2.1 is the quadratic misspecification model. The total number of subjects in the study is n = 1, 000. We present the proportion of rejecting H0 for 10 different sample sizes ranging between 50 and 1,000. For each size, the samples are randomly drawn from the original dataset. The result is displayed in Figure 3.9. We see that Cn performs almost identically to Stukel statistic at each sample size. Increasing sample size leads to a considerable increase in rejection rate in both Cn and Stukel compared to Pearson and Deviance whose power gain is much smaller. Including all the data (n=1,000) increases the power of Cn and Stukel to 0.7091, with both Deviance and Pearson statistics lagging behind. In this example, Deviance doesn’t show the inflated type one error rate since the response variables Yi has large counts and thus the large sample condition holds for Deviance. 3.3.4 Negative Binomial Example 1 The previous 2 examples demonstrate the performance of Cn compared to Deviance, Pearson, and Stukel statistic in Poisson regression, where the link function is canon- ical. We now present two numerical example to examine the Cn GOF in negative binomial regression with non-canonical link. The derivation of the test statistic for 79 Table 3.9: Selected variables description in the German Credit data (n=1,000) Symbol Codes/Values Variable description y Months Duration of Credits a Years Age c US Dollars Credit Amount Table 3.10: Proportion H0 rejected at the α = 0.05 using sample size of 1000 with 10000 replications using the German Credit data Model Test statistics Proportion H0 rejected Under H0: German 1.0 Cn 0.0475 P earson 0.0475 Deviance 0.0716 Stukel 0.0484 80 Figure 3.2: Proportion H0 rejected at the α = 0.05 over increase sample size with 10000 replications using the German Credit data C_n Pearson 0.8 Deviance Stukel Power 0.4 0.0 200 400 600 800 1000 Sample Size non-canonical link is analogous to the case of canonical link. The set of covariates were from Bad Health dataset from German National Health Interview and Exam- ination Survey (GNHIES98) with the ”German Health Interview and Examination Survey for Adults”(DEGS) collected by the Robert Koch Institute [107]. The aim of the DEGS survey was to repeatedly collect representative data on the health sta- tus, health-related behavior, healthcare and living conditions of adults residing in Germany. Given the amount of overdispersion in the data, employing a negative bi- nomial model appears to be an appropriate strategy. Table 3.11 provides the details of the dataset. The response variable Yi is the number of doctor visits during 1998. Models used to generate Yi are: 81 log(µi) = 0.404 + 1.108bi − 0.0069ai DEGS 3.0 2 log(µi) = 0.404 + 1.108bi − 0.0069ai + K × ai DEGS 3.1 Where DEGS 3.0 is the null model. Independent variables are the age of the pa- tients(a), and dichotomous covariate indicating whether patient claims to be in bad health or not. DEGS 3.1 is quadratic misspecification model, where K is the amount of misspecification and takes the values in the range from 0 to 0.0011 by 0.0001. Fig- ure 3.3 displays the summary rejection rate of the 4 GOF Cn, Pearson, Deviance, and Stukel statistics. Similarly to what we observed in the Affair data, Cn and Pearson, and Stukel statistics appear to have a reasonable type one error, and all demonstrate an improvement in power as K increases. Obviously, Cn and Stukel capture the misspecification fairly quickly, as K = 0.0011, Cn rejects H0 90% of the time. It should be noted that the means of the response Yi are small, explaining the erratic behavior of Deviance. Thus in the current negative binomial regression example we reach similar conclusions as in the Poisson regression under canonical link. 3.3.5 Negative Binomial Example 2 In the last example we investigate the sensitivity of Cn to the link function misspeci- fication. All the models considered so far use the log transformation of the expected values, but other choices are possible for negative binomial regression. In fact, any 82 Table 3.11: Selected variables description in the Bad Health data Symbol Codes/Values Variable description y Visits Number of visits to doctor b 0 = No, 1 = Yes Patient claims to be in bad health a Years Age of patient Table 3.12: Proportion H0 rejected at the α = 0.05 using sample size of 1127 with 10000 replications using the Bad Health data Model Test statistics Proportion H0 rejected Under H0: DEGS 1.0 Cn 0.0419 P earson 0.0370 Deviance 0.9771 Stukel 0.0506 83 Figure 3.3: Proportion H0 rejected at the α = 0.05 in quadratic models with sample size of 1127 with 10000 replications in the Bad Health data 1.0 C_n Pearson 0.8 Deviance Stukel 0.6 Power 0.4 0.2 0.0 0e+00 2e−04 4e−04 6e−04 8e−04 1e−03 Misspecification K transformation that maps mean into the real line could be used to produce a nega- tive binomial outcome, as long as the transformation is one-to-one, continuous and differentiable. Covariates used in this example came from the Fishing data. The original data were from Bailey et al. [108], the goal of their study was to explore how certain deep-sea fish populations were impacted when commercial fishing began in deeper water locations. There are 147 observations in the dataset. Table 3.13 displays the variables. 84 Table 3.13: Selected variables description in the Fishing data Symbol Codes/Values Variable description y Counts Total fish counted per site d Depth Mean water depth per site f Density Folage density index Models used to generate outcome variables Yi are: log(µi) = 5.466 + 80.44di − 0.0003fi Fishing 4.0 √ µi = 5.466 + 80.44di − 0.0003fi Fishing 4.1 The response here is the total fish counted per site. Potential explanatory variables are the mean water depth(d), and forlage density(f). We simulated the response Yi from Fishing 4.0 with the most commonly used log transformation, and from model 4.1 using square root link. Table 3.14 displays the result. Neither Deviance nor Pear- son statistic appear effective in detecting false link function. They lack statistical power and do not attain nominal significance level even with serious misspecifica- tion. However both Cn and Stukel show the capability of detecting misspecified link function, even with moderate sample size n = 147. 85 Table 3.14: Proportion H0 rejected at the α = 0.05 using sample size of 147 with 10000 replications in sqrt link model using the Fishing data Model Test statistics Proportion H0 rejected Under H0: Fishing 4.0 Cn 0.0413 P earson 0.0166 Deviance 0 Stukel 0.0562 Under Ha: Fishing 4.1 Cn 0.8223 P earson 0.0113 Deviance 0 Stukel 0.8223 3.4 Model Selection Example 3.4.1 Model Selection Example for Binary Variable As our final example we present an application of the SSRs statistic Cn to the model selection problem by comparing its fit under different link functions. The dataset was taken from the National Cancer Institute’s Health Information National Trends Survey (HINTS 2007) which is a national effort to monitor efforts in public health knowledge dissemination among general public and its effects on cancer prevention and monitoring. More detailed information on the HINTS 2007 survey can be found elsewhere [109]. For our current example a subset of HINTS data was selected with three variables of interest described in Table 3.15. Here the variable cancer is the 86 Table 3.15: Selected variables description in HINTS data Symbol Codes/Values Variable description cancer 0 = Absence, 1 = Presence Previous Cancer Diagnosis age Years Age race 1= White, 2= Black, 3=Unknown Race outcome of interest, with age and race being two explanatory variables. Three samples (each of size 100) were randomly drawn to assess reproducibility of the performance of Cn for small-to-moderate sample sizes. For each sample, the GOF test using Cn under three different models represented by the link functions: logit, probit and complementary-log-log was performed and the asymptotic p-values were used for data-based models discrimination. Table 3.16 presents the respective values of the statistic Cn and the corresponding p-values under each link. As seen from the table, the test statistic Cn gives evidence against the probit link but not against the remaining two links (which very consistently gave similar values of Cn across all three samples). Consequently, our analysis indicates that the model with the probit link is not supported by the data, whereas the other two models possibly are. 3.4.2 Model Discrimination Example for Count Variable Our last example demonstrates the implementation of Cn statistic to a model selec- tion problem by comparing the fit under Poisson and negative binomial models. The 87 Table 3.16: The results of Cn and its p-values under each model using the HINTS data Sample 1 Sample 2 Sample 3 Link Function Cn P-value Cn P-value Cn P-value Logit 1.08 0.28 -0.498 0.62 -1.06 0.29 Probit 3.78 0.00016 -2.16 0.03 -4.07 0.00005 Cloglog 1.08 0.28 -0.474 0.63 -1.16 0.25 dataset is pertussis public data downloaded from Centers for Disease Control and Prevention (CDC) and discussed in Chapter 2. Recall that the outcome variable is the yearly number of pertussis cases, and 4 corresponding covariates are selected. One is the estimated DTP/DTaP/DT yearly vaccination coverage among children enrolled in kindergarten, by state from CDC Morbidity and Mortality Weekly Report (MMWR)[57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73]. The explanatory variable is medicare average yearly enrollment by state downloaded from government resources Centers for Medicare & Medicaid Services (CMS) [74] session of Medicare coverage database [75]. Two time related variable are added in model fitting since it is essentially longitudinal data, and we expect there is autocorrelation over time. More details of the variables can be found in Table 3.17. The statistic Cn and p−values are evaluated under Poisson and negative bimonial models using log link. Table 3.18 presents the results. 88 Table 3.17: Pertussis data Symbol Codes/Values Variable description y Counts Number of pertussis cases by state and year m Rate Medicare average enrollment by state and year v Rate Vaccination coverage by state and year t Linear Time as linear predictor tt Quadratic Squared time Table 3.18: The results of Cn and its p−values under Poisson and negative bimonial model using Pertussis data Model Cn p−value Poisson -14.63622 < 0.0001 Negative Binomial 3.073157 0.0035 It appears that Cn provides evidence against both models under log link. This indicates that neither Poisson nor negative bimonial is appropriate for the data. 3.5 Discussion In this chapter, we proposed a new GOF test for binary/discrete GLMs based on the sum of standardized residuals. We have shown that the statistic Cn follows a normal distribution under certain regularity conditions (Theorem 1). We see that, even for 89 moderate sample sizes Cn holds nominal type one error under H0. It also exhibits satisfactory power and outperforms alternative GOF tests across different scenarios in Poisson and negative binomial regression models examined here. Based on our numerical example, we conclude that Cn is a simple and well-performing GOF test that can be used in a number of circumstances in GLMs setting. However, it should be noted that there are some cases where Cn does not work, as we can see for instance in the Gaussian linear regression. In general, as already 0 T 1 indicated earlier, Cn is degenerate whenever ψ (Zi β) ∝ √ . Note that in case V (µi) 2 yµ−µ2/2 of the Gaussian distribution, with p.d.f. f(y|µ, σ ) = h(y)exp{ σ2 }, the mean and variance are, respectively, E(Y ) = µ(θ) = θ = ZT β and Var(Y ) = V (µ) = σ2. T From the formulas for νn and Ω, n ∂µ(θi) n T 1 X ∂β 1 X Z νT = 0 = √ i , n n p n 2 i=1 V (µi) i=1 σ n 2 n T 1 X ∂log f(Yi) 1 X Z Zi Ω = = i . n T n σ2 i=1 ∂β∂β i=1 T −1 2 and thus νn Ω νn = 1, so that σn = 0 in Theorem 1. 90 Chapter 4: Future Work In this work, we have derived a model for spatial dependency where longitudinal data are collected at multiple locations by incorporating an autoregressive process into the MVP framework to create a dynamic MVP or DMVP model. One extension of the DMVP framework is to account for overdispersion, where greater variability in a data set is present than expected in a given Poisson outcome. One can achieve this by considering a negative binomial distribution in the observation equation instead of Poisson. The observation equation of the negative binomial outcome then becomes X Xi,t = Yi,t + ξij,t, j6=i pi,tri,t Yi,t ∼ NB(ri,t; pi,t); E(Yi,t) = µi,t = , 1 − pi,t w T X (4.1) log(µi,t) = Zi,t βt + b0t + bl(t − τl)+, l=1 πij,tκij,t ξij,t ∼ NB(κij,t; πij,t); E(ξij,t) = ηij,t = , 1 − πij,t T log(ηij,t) = Dij,t γt. 1 1 Where r and κ are referred as ”dispersion parameter” or ”shape parameter”. If r and κ increase towards infinity, the negative binomial can be approximated by the 91 Poisson distribution. Temporal dependence can be incorporated in the same way as for DMVP, that is, with the autoregressive model. For the analysis of area-level count data, it is possible that there is an excessive number of areas with 0 count, especially if we are interested in mortality data at the small area level such as county and census tract. A popular approach to deal with such data is to use a zero-inflated Poisson (ZIP) regression model. The ZIP model combines the standard Poisson distribution with a mixture of point mass at zero. One should also consider the case that non-zero counts are overdispersed in relation to the Poisson distribution. In the presence of both ZIP and overdispersion, zero-inflated negative binomial (ZINB) model might be an appropriate choice. However, having a lot of zeros does not necessarily mean that we need a zero-inflated model. The difference between fitting a conventional negative binomial and ZIBN is relatively small computationally and one can conduct a likelihood ratio test (or use Cn-based model selection) to compare the results and choose the appropriate model. After joinpoints are incorporated into the model, we need to carefully investigate the choice of the initial number of knots h. We need large value of h so that the true joinpoints are well approximated. On the other hand, the model might become un-identifiable if h is chosen to be too large. Hence searching for the pre-specified h requires first some practically feasible upper bound estimate. We proposed Cn statistics that can be applied to detect lack of fit for logistic, Poisson, and Negative Binomial regression. The fundamental assumption is the independence 92 of the random variables so that law of large number can be used to derive the asymptotic distribution. However, responses involved in public health studies is a series of responses measured repeatedly over time and therefore are not independent. Recently, there has been some efforts to develop GOF tests for longitudinal models from 2 prospectives [110]. 1. Generalizing methods for independent variables such as HL test. 2. Formulating new tests using generalized estimating equations (GEE) estimates. However, most of these research are limited for binary outcomes only. One import direction for future study is to generalize our test statistic Cn to spatial and temporal dependent data. Given the satisfying performance of Cn, it is not unrealistic to suspect that the test derived for dependent data would provide a good measurement to detect lack of fit. However, the difficulty of developing such a test statistic is to find a normalizing constant to obtain its asymptotic distribution. 93 Bibliography [1] L. Chen, Y. Sui, C. Song, and G. A. Rempala, “The sum of standardized residuals: Goodness- of-fit test for binary response models,” Statistics in medicine, 2017. [2] R. S. Greenberg and D. Kleinbawm, “Mathematical modeling strategies for the analysis of epidemiologic research,” Annual review of public health, vol. 6, no. 1, pp. 223–245, 1985. [3] D. Bernoulli, “Essai d’une nouvelle analyse de la mortalit´ecaus´eepar la petite v´erole,et des avantages de l’inoculation pour la pr´evenir,” Histoire de l’Acad., Roy. Sci.(Paris) avec Mem, pp. 1–45, 1760. [4] C. I. Siettos and L. Russo, “Mathematical modeling of infectious disease dynamics,” Viru- lence, vol. 4, no. 4, pp. 295–306, 2013. [5] “Pertussis (Whooping Cough) Surveillance and Reporting,” https://www.cdc.gov/pertussis/ surv-reporting.html, accessed: 2018-04-12. [6] “Pertussis Public Health Professionals,” https://www.cdc.gov/pertussis/php.html, accessed: 2018-04-12. [7] M. Gambhir, C. Bozio, J. J. O’Hagan, A. Uzicanin, L. E. Johnson, M. Biggerstaff, and D. L. Swerdlow, “Infectious disease modeling methods as tools for informing response to novel influenza viruses of unknown pandemic potential,” Clinical Infectious Diseases, vol. 60, no. suppl 1, pp. S11–S19, 2015. [8] R. S. Ostfeld, G. E. Glass, and F. Keesing, “Spatial epidemiology: an emerging (or re- emerging) discipline,” Trends in ecology & evolution, vol. 20, no. 6, pp. 328–336, 2005. [9] T. W. Anderson, T. W. Anderson, T. W. Anderson, T. W. Anderson, and E.-U. Math´ematicien, An introduction to multivariate statistical analysis. Wiley New York, 1958, vol. 2. [10] D. Mahamunulu, “A note on regression in the multivariate poisson distribution,” Journal of the American Statistical Association, vol. 62, no. 317, pp. 251–258, 1967. 94 [11] A. M’Kendrick, “Applications of mathematics to medical problems,” Proceedings of the Ed- inburgh Mathematical Society, vol. 44, pp. 98–130, 1925. [12] D. I. Inouye, E. Yang, G. I. Allen, and P. Ravikumar, “A review of multivariate distribu- tions for count data derived from the poisson distribution,” Wiley Interdisciplinary Reviews: Computational Statistics, vol. 9, no. 3, 2017. [13] E. G. Tsionas, “Bayesian analysis of the multivariate poisson distribution,” Communications in Statistics-Theory and Methods, vol. 28, no. 2, pp. 431–451, 1999. [14] Tsionas, Efthymios G, “Bayesian multivariate poisson regression,” Communications in Statistics-Theory and Methods, vol. 30, no. 2, pp. 243–255, 2001. [15] D. Karlis, “An em algorithm for multivariate poisson distribution and related models,” Jour- nal of Applied Statistics, vol. 30, no. 1, pp. 63–77, 2003. [16] D. Karlis and L. Meligkotsidou, “Multivariate poisson regression with covariance structure,” Statistics and Computing, vol. 15, no. 4, pp. 255–265, 2005. [17] X. Pedeli and D. Karlis, “A bivariate inar (1) process with application,” Statistical modelling, vol. 11, no. 4, pp. 325–349, 2011. [18] T. Aktekin, N. G. Polson, and R. Soyer, “Sequential bayesian analysis of multivariate count data,” arXiv preprint arXiv:1602.01445, 2016. [19] T. McCue, E. Carruthers, J. Dawe, S. Liu, A. Robar, and K. Johnson, “Evaluation of gen- eralized linear model assumptions using randomization,” Unpublished manuscript. Retrieved fro m http://www. mun. ca/biology/dschneider/b7932/B7932Final10Dec2008. pdf, 2008. [20] P. McCullagh, “Generalized linear models,” European Journal of Operational Research, vol. 16, no. 3, pp. 285–292, 1984. [21] N. L. Johnson, S. Kotz, and N. Balakrishnan, Discrete multivariate distributions. Wiley New York, 1997, vol. 165. [22] D. Gamerman and H. S. Migon, “Hierarchical dynamic models,” Journal of the Royal Statis- tical Society, vol. 55, no. 3, pp. 629–642, 1993. [23] D. M. Hawkins, “Fitting multiple change-point models fitting multiple change-point models to data,” Computational Statistics Data Analysis, vol. 37, pp. 323–341, 2001. [24] R. Gill, F. G. Holden, , and G. A. Rempala, “Penalized regression for anomaly detection in cohort mortality patterns.” 95 [25] M. Czajkowski, R. Gill, and G. Rempala, “Model selection in logistic joinpoint regression with applications to analyzing cohort mortality patterns,” STATISTICS IN MEDICINE, vol. 27, pp. 1508–1526, 8 2008. [26] J. L. L´opez-Campos, M. Ruiz-Ramos, and J. B. Soriano, “Mortality trends in chronic ob- structive pulmonary disease in europe, 1994–2010: a joinpoint regression analysis,” The lancet Respiratory medicine, vol. 2, no. 1, pp. 54–62, 2014. [27] M. Malvezzi, P. Bertuccio, F. Levi, C. La Vecchia, and E. Negri, “European cancer mortality predictions for the year 2013,” Annals of Oncology, p. mdt010, 2013. [28] M. Malvezzi, P. Bertuccio, T. Rosso, M. Rota, F. Levi, C. La Vecchia, and E. Negri, “Euro- pean cancer mortality predictions for the year 2015: does lung cancer have the highest death rate in eu women?” Annals of Oncology, vol. 26, no. 4, pp. 779–786, 2015. [29] L. Pan, H. M. Blanck, B. Sherry, K. Dalenius, and L. M. Grummer-Strawn, “Trends in the prevalence of extreme obesity among us preschool-aged children living in low-income families, 1998-2010,” Jama, vol. 308, no. 24, pp. 2563–2565, 2012. [30] R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Sta- tistical Society, vol. 58, no. 1, pp. 267–288, 1996. [31] T. PARK and G. CASELLA, “The bayesian lasso,” Journal of the American Statistical As- sociation, vol. 103, no. 482, pp. 681–686, 2008. [32] W. R. Gilks and P. Wild, “Adaptive rejection sampling for gibbs sampling,” Applied Statistics, pp. 337–348, 1992. [33] W. R. Gilks, N. Best, and K. Tan, “Adaptive rejection metropolis sampling within gibbs sampling,” Applied Statistics, pp. 455–472, 1995. [34] G. Rozhnova and A. Nunes, “Modelling the long-term dynamics of pre-vaccination pertussis,” Journal of the Royal Society Interface, vol. 9, no. 76, pp. 2959–2970, 2012. [35] P. Pesco, P. Bergero, G. Fabricius, and D. Hozbor, “Modelling the effect of changes in vaccine effectiveness and transmission contact rates on pertussis epidemiology,” Epidemics, vol. 7, pp. 13–21, 2014. [36] F. R. Mooi, “Bordetella pertussis and vaccination: the persistence of a genetically monomor- phic pathogen,” Infection, Genetics and Evolution, vol. 10, no. 1, pp. 36–49, 2010. [37] K. M. Bisgard, F. B. Pascual, K. R. Ehresmann, C. A. Miller, C. Cianfrini, C. E. Jennings, C. A. Rebmann, J. Gabel, S. L. Schauer, and S. M. Lett, “Infant pertussis: who was the source?” The Pediatric infectious disease journal, vol. 23, no. 11, pp. 985–989, 2004. 96 [38] “Pertussis Causes and Transmission,” https://www.cdc.gov/pertussis/about/ causes-transmission.html, accessed: 2018-04-12. [39] S. C. A. Joseph J Bocka, Bryon K McNeil, “Clinical infectious diseases.” Drug and Diseases: Pediatrics: General Medicine, 2017. [40] “Pertussis Clinician Features and Stages,” https://www.cdc.gov/pertussis/clinical/features. html, accessed: 2018-04-12. [41] R. Wallace, “Analysis. epidemiology and public healthin maxcy rosenau last public health and preventive medicine,” 2008. [42] “Pertussis Complications in Babies and Children,” https://www.cdc.gov/pertussis/about/ complications.html, accessed: 2018-04-12. [43] M. Tanaka, C. R. Vitek, F. B. Pascual, K. M. Bisgard, J. E. Tate, and T. V. Murphy, “Trends in pertussis among infants in the united states, 1980-1999,” Jama, vol. 290, no. 22, pp. 2968–2975, 2003. [44] W. H. Organization et al., “Pertussis vaccines: Who position paper?august 2015= note de synth`ese: Position de l?oms concernant les vaccins anticoquelucheux?aoˆut2015,” Weekly Epidemiological Record= Relev´e´epid´emiologique hebdomadaire, vol. 90, no. 35, pp. 433–458, 2015. [45] World Health Organization and others, “The immunological basis for immunization series: module 4: pertussis-update 2017,” 2017. [46] T. H. Skoff, C. Kenyon, N. Cocoros, J. Liko, L. Miller, K. Kudish, J. Baumbach, S. Zansky, A. Faulkner, and S. W. Martin, “Sources of infant pertussis infection in the united states,” Pediatrics, vol. 136, no. 4, pp. 635–641, 2015. [47] A. M. Wendelboe, E. Njamkepo, A. Bourillon, D. D. Floret, J. Gaudelus, M. Gerber, E. Grim- prel, D. Greenberg, S. Halperin, J. Liese et al., “Transmission of bordetella pertussis to young infants,” The Pediatric infectious disease journal, vol. 26, no. 4, pp. 293–299, 2007. [48] J. D. Cherry, “Historical review of pertussis and the classical vaccine,” Journal of Infectious Diseases, vol. 174, no. Supplement 3, pp. S259–S263, 1996. [49] J. D. Cherry, P. A. Brunell, G. S. Golden, and D. T. Karzon, “Report of the task force on pertussis and pertussis immunization?1988,” Pediatrics, vol. 81, no. 6, pp. 933–984, 1988. [50] T. Madsen, “Vaccination against whooping cough,” Journal of the American Medical Asso- ciation, vol. 101, no. 3, pp. 187–188, 1933. 97 [51] J. D. Cherry, “The history of pertussis (whooping cough); 1906–2015: facts, myths, and misconceptions,” Current Epidemiology Reports, vol. 2, no. 2, pp. 120–130, 2015. [52] S. Mattoo and J. D. Cherry, “Molecular pathogenesis, epidemiology, and clinical manifesta- tions of respiratory infections due to bordetella pertussis and other bordetella subspecies,” Clinical microbiology reviews, vol. 18, no. 2, pp. 326–382, 2005. [53] A. Hoc, “Group for the study of pertussis vaccines. placebocontrolled trial of two acellular pertussis vaccines in sweden–protective efficacy and adverse events,” Lancet, vol. 1, pp. 955– 60, 1988. [54] P. Olin, F. Rasmussen, L. Gustafsson, H. O. Hallander, H. Heijbel et al., “Randomised controlled trial of two-component, three-component, and five-component acellular pertussis vaccines compared with whole-cell pertussis vaccine,” The Lancet, vol. 350, no. 9091, pp. 1569–1577, 1997. [55] L. Gustafsson, L. Hessel, J. Storsaeter, and P. Olin, “Long-term follow-up of swedish children vaccinated with acellular pertussis vaccines at 3, 5, and 12 months of age indicates the need for a booster dose at 5 to 7 years of age,” Pediatrics, vol. 118, no. 3, pp. 978–984, 2006. [56] “CDC Wide-ranging ONline Data for Epidemiologic Research (CDC WONDER) ,” https: //wonder.cdc.gov/, accessed: 2018-04-12. [57] C. for Disease Control, P. (CDC et al., “National, state, and urban area vaccination cover- age levels among children aged 19-35 months–united states, 2000.” MMWR. Morbidity and mortality weekly report, vol. 50, no. 30, p. 637, 2001. [58] L. Barker, E. Luman, Z. Zhao, P. Smith, R. Linkins, J. Santoli, L. Rodewald, M. McCauley et al., “National, state, and urban area vaccination coverage levels among children aged 19-35 months–united states, 2001.” MMWR. Morbidity and mortality weekly report, vol. 51, no. 30, p. 664, 2002. [59] Centers for Disease Control and Prevention (CDC and others), “National, state, and urban area vaccination levels among children aged 19-35 months–united states, 2002.” MMWR. Morbidity and mortality weekly report, vol. 52, no. 31, p. 728, 2003. [60] Centers for Disease Control and Prevention (CDC and others), “National, state, and urban area vaccination coverage among children aged 19-35 months–united states, 2003.” MMWR. Morbidity and mortality weekly report, vol. 53, no. 29, p. 658, 2004. [61] Centers for Disease Control and Prevention (CDC and others), “National, state, and urban area vaccination coverage among children aged 19-35 months–united states, 2004.” MMWR. Morbidity and mortality weekly report, vol. 54, no. 29, p. 717, 2005. 98 [62] Centers for Disease Control and Prevention (CDC and others), “National, state, and urban area vaccination coverage among children aged 19-35 months–united states, 2005.” MMWR. Morbidity and mortality weekly report, vol. 55, no. 36, p. 988, 2006. [63] Centers for Disease Control and Prevention (CDC and others), “National, state, and local area vaccination coverage among children aged 19-35 months–united states, 2006.” MMWR. Morbidity and mortality weekly report, vol. 56, no. 34, p. 880, 2007. [64] Centers for Disease Control and Prevention (CDC and others), “National, state, and local area vaccination coverage among children aged 19-35 months–united states, 2007.” MMWR. Morbidity and mortality weekly report, vol. 57, no. 35, p. 961, 2008. [65] Centers for Disease Control and Prevention (CDC and others), “National, state, and local area vaccination coverage among children aged 19-35 months-united states, 2008.” MMWR. Morbidity and mortality weekly report, vol. 58, no. 33, p. 921, 2009. [66] Centers for Disease Control and Prevention (CDC and others), “National, state, and local area vaccination coverage among children aged 19-35 months—united states, 2009.” MMWR. Morbidity and mortality weekly report, vol. 59, no. 36, p. 1171, 2010. [67] Centers for Disease Control and Prevention (CDC and others), “Office-related antibiotic prescribing for persons aged? 14 years–united states, 1993-1994 to 2007-2008.” MMWR. Morbidity and mortality weekly report, vol. 60, no. 34, p. 1153, 2011. [68] Centers for Disease Control and Prevention (CDC and others), “Vaccination coverage among children in kindergarten–united states, 2009-10 school year.” MMWR. Morbidity and mortal- ity weekly report, vol. 60, no. 21, p. 700, 2011. [69] Centers for Disease Control and Prevention (CDC and others), “Vaccination coverage among children in kindergarten–united states, 2011-12 school year.” MMWR. Morbidity and mortal- ity weekly report, vol. 61, no. 33, p. 647, 2012. [70] Centers for Disease Control and Prevention (CDC and others), “Vaccination coverage among children in kindergarten-united states, 2012-13 school year.” MMWR. Morbidity and mortality weekly report, vol. 62, no. 30, p. 607, 2013. [71] R. Seither, S. Masalovich, C. L. Knighton, J. Mellerson, J. A. Singleton, and S. M. Greby, “Vaccination coverage among children in kindergarten-united states, 2013-14 school year.” Morbidity and mortality weekly report, vol. 63, no. 41, pp. 913–920, 2014. [72] R. Seither, K. Calhoun, C. L. Knighton, J. Mellerson, S. Meador, A. Tippins, S. M. Greby, and V. Dietz, “Vaccination coverage among children in kindergarten?united states, 2014–15 school year,” MMWR Morb Mortal Wkly Rep, vol. 64, no. 33, pp. 897–904, 2015. 99 [73] R. Seither, “Vaccination coverage among children in kindergarten?united states, 2015–16 school year,” MMWR. Morbidity and mortality weekly report, vol. 65, 2016. [74] “Centers for Medicare and Medicaid Services,” https://www.cms.gov/, accessed: 2018-04-12. [75] “Centers for Medicare and Medicaid Services Historical Enrollment Data,” https://www.cms. gov/medicare-coverage-database/, accessed: 2018-04-12. [76] A. K. Lugn´er, S. D. Mylius, and J. Wallinga, “Dynamic versus static models in cost- effectiveness analyses of anti-viral drug therapy to mitigate an influenza pandemic,” Health economics, vol. 19, no. 5, pp. 518–531, 2010. [77] W. Edmunds, G. Medley, and D. Nokes, “Evaluating the cost-effectiveness of vaccination programmes: a dynamic perspective,” Statistics in medicine, vol. 18, no. 23, pp. 3263–3282, 1999. [78] P. Beutels, W. J. Edmunds, F. Anto˜nanzas,G. A. De Wit, D. Evans, R. Feilden, A. M. Fendrick, G. M. Ginsberg, H. A. Glick, E. Mast et al., “Economic evaluation of vaccination programmes,” Pharmacoeconomics, vol. 20, no. 1, pp. 1–7, 2002. [79] M. Brisson and W. Edmunds, “Impact of model, methodological, and parameter uncertainty in the economic analysis of vaccination programs,” Medical decision making, vol. 26, no. 5, pp. 434–446, 2006. [80] M. Drummond, C. Chevat, and M. Lothgren, “Do we fully understand the economic value of vaccines?” Vaccine, vol. 25, no. 32, pp. 5945–5957, 2007. [81] N. M. Ferguson, D. A. Cummings, S. Cauchemez, C. Fraser, S. Riley, A. Meeyai, S. Iam- sirithaworn, and D. S. Burke, “Strategies for containing an emerging influenza pandemic in southeast asia,” Nature, vol. 437, no. 7056, p. 209, 2005. [82] I. M. Longini, A. Nizam, S. Xu, K. Ungchusak, W. Hanshaoworakul, D. A. Cummings, and M. E. Halloran, “Containing pandemic influenza at the source,” Science, vol. 309, no. 5737, pp. 1083–1087, 2005. [83] N. Ferguson, “Poverty, death, and a future influenza pandemic,” The Lancet, vol. 368, no. 9554, pp. 2187–2188, 2006. [84] T. C. Germann, K. Kadau, I. M. Longini, and C. A. Macken, “Mitigation strategies for pandemic influenza in the united states,” Proceedings of the National Academy of Sciences, vol. 103, no. 15, pp. 5935–5940, 2006. [85] M. Lipsitch, T. Cohen, M. Murray, and B. R. Levin, “Antiviral resistance and the control of pandemic influenza,” PLoS medicine, vol. 4, no. 1, p. e15, 2007. 100 [86] D. Hosmer, S. Lemeshow, and R. Sturdivant, Applied Logistic Regression, third edition ed. Wiley, 2013. [87] B. Zheng, “Summarizing the goodness of fit of generalized linear models for longitudinal data,” Statistics in medicine, vol. 19, no. 10, pp. 1265–1275, 2000. [88] D. W. Hosmer and S. Lemesbow, “Goodness of fit tests for the multiple logistic regression model,” Communications in statistics-Theory and Methods, vol. 9, no. 10, pp. 1043–1069, 1980. [89] T. A. Stukel, “Generalized logistic models,” Journal of the American Statistical Association, vol. 83, no. 402, pp. 426–431, 1988. [90] J. Copas, “Unweighted sum of squares test for proportions,” Applied Statistics, pp. 71–80, 1989. [91] H. White, “Maximum likelihood estimation of misspecified models,” Econometrica: Journal of the Econometric Society, pp. 1–25, 1982. [92] C. Orme, “The calculation of the information matrix test for binary data models,” The Manchester School, vol. 56, no. 4, pp. 370–376, 1988. [93] W. K. Newey, “Maximum likelihood specification testing and conditional moment tests,” Econometrica: Journal of the Econometric Society, pp. 1047–1070, 1985. [94] R. Davidson and J. G. MacKinnon, “Convenient specification tests for logit and probit mod- els,” Journal of Econometrics, vol. 25, no. 3, pp. 241–262, 1984. [95] F. A. G. Windmeijer, “The asymptotic distribution of the sum of weighted squared residuals in binary choice models,” Statistica Neerlandica, vol. 44, no. 2, pp. 69–78, 1990. [96] K. Pearson, “X. on the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, vol. 50, no. 302, pp. 157–175, 1900. [97] G. K. Smyth, “Pearson’s goodness of fit statistic as a score test statistic,” Lecture Notes- Monograph Series, pp. 115–126, 2003. [98] G. Osius and D. Rojek, “Normal goodness-of-fit tests for multinomial models with large degrees of freedom,” Journal of the American Statistical Association, vol. 87, no. 420, pp. 1145–1152, 1992. [99] D. Williams, “Generalized linear model diagnostics using the deviance and single case dele- tions,” Applied Statistics, pp. 181–191, 1987. 101 [100] L. Wan and G. Rempala, “Glm note,” 2016. [101] T. Amemiya, Advanced Econometrics. Harvard University Press, 1985. [102] J. Shao, “Mathematical statistics,” 2003. [103] C. S. Almond, A. Y. Shin, E. B. Fortescue, R. C. Mannix, D. Wypij, B. A. Binstadt, C. N. Duncan, D. P. Olson, A. E. Salerno, J. W. Newburger et al., “Hyponatremia among runners in the boston marathon,” New England Journal of Medicine, vol. 352, no. 15, pp. 1550–1556, 2005. [104] M. Lichman, “UCI machine learning repository,” 2013. [Online]. Available: http: //archive.ics.uci.edu/ml [105] R. C. Fair, “A theory of extramarital affairs,” Journal of Political Economy, vol. 86, no. 1, pp. 45–61, 1978. [106] “German Credit Data,” ftp://ftp.ics.uci.edu/pub/machine-learning-databases/statlog/, ac- cessed: 2018-04-12. [107] B.-M. Bellach, H. Knopf, and W. Thefeld, “The german health survey. 1997/98,” Gesund- heitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)), vol. 60, pp. S59–68, 1998. [108] A. F. Zuur, J. M. Hilbe, and E. N. Ieno, A Beginner’s Guide to GLM and GLMM with R: A Frequentist and Bayesian Perspective for Ecologists. Highland Statistics Limited, 2013. [109] D. Cantor, K. Coa, S. Crystal-Mansour, T. Davis, S. Dipko, and R. Sigman, “Health infor- mation national trends survey (hints) 2007: Final report. 2009,” 2011. [110] K.-C. Lin, “Goodness-of-fit tests for modeling longitudinal ordinal data,” Computational Statistics & Data Analysis, vol. 54, no. 7, pp. 1872–1880, 2010. 102 Appendix A: The Sum of Standardized Residuals: Goodness of Fit Test for Binary Response Model 103 Received: 1 June 2017 Revised: 31 January 2018 Accepted: 1 February 2018 DOI: 10.1002/sim.7644 RESEARCH ARTICLE The sum of standardized residuals: Goodness-of-fit test for binary response models Lu Chen1 Yihan Sui1 Chi Song1 Grzegorz A. Rempala1,2 1Division of Biostatistics, The Ohio State University, Columbus, OH 43210, U.S.A. We propose a new goodness-of-fit statistic for evaluating generalized linear mod- 2Mathematical Biosciences Institute, The els with binary responses on the basis of the sum of standardized residuals. We Ohio State University, Columbus, OH derive the asymptotic distribution of the sum of standardized residuals statistic 43210, U.S.A. and argue that, despite its relative simplicity, it typically outperforms many of Correspondence the more sophisticated currently used goodness-of-fit statistics. Grzegorz A. Rempala, Mathematical Biosciences Institute, The Ohio State KEYWORDS University, Columbus, OH 43210, U.S.A. binary response model, cohort studies, goodness-of-fit test, model residuals Email: [email protected] Funding information National Science Foundation, Grant/Award Number: DMS 1440386 1 INTRODUCTION The binary response model1 is a popular method of analyzing binary data in the regression setting. For a binary random variable Yi ∈{0, 1}, we assume that its conditional probability given the covariates is pi = P(Yi = 1 xi), which may be represented as ⊤ 𝑝i = P(Yi = 1 xi)=F(xi 𝛽), | (1) where 𝛽 is the k-vector of coefficients and F is a distribution function. Some popular choices of F include normal (probit model) or logistic (logit model) distributions. | After the model is fitted by selecting the estimated values of 𝛽, we usually want to assess its goodness of fit (GOF), that is, determine whether the model fits the data in some appropriate sense.2 We will review several currently used GOF test statistics in the next section. We also propose a new statistic based on the sum of standardized residuals or SSRs and denoted throughout the paper by Cn. We describe its asymptotic distribution in Section 2. In Section 3, we illustrate, via numerical examples, the practical applications of the asymptotic result to some finite data samples and compare Cn's performance with that of other statistics on a set of examples taken from public health studies. 1.1 Some currently used goodness-of-fit statistics 1.1.1 Hosmer-Lemeshow statistic In 1980, Hosmer and Lemeshow3 proposed a grouping-based method to measure the GOF for the binary response model. ⊤ Specifically, the data are grouped according to the 𝑝̂i's, the estimated values of pi defined by 𝑝̂i = F(xi 𝛽̂),where𝛽̂ is the data-based estimate of 𝛽, typically its maximum likelihood estimator (MLE). The observed and expected frequency are calculated for each group and compared using the Pearson chi-square statistic, to obtain g 2 2 (O1k − Ê1k) (O0k − Ê0k) HL = + , Ê Ê k=1 [ 1k 0k ] ∑ Statistics in Medicine. 2018;1–10. wileyonlinelibrary.com/journal/sim Copyright © 2018 John Wiley & Sons, Ltd. 1 104 2 CHEN ET AL. ck where k denotes the kth group, k 1 2 3 g. O1k Y is the observed number of events (Yj 1) in the kth = , , , … , = 𝑗=1 𝑗 = ck group, where ck is the number of covariate patterns in the kth group. Ê1k = m is the estimated expected number of ∑ 𝑗=1 𝑗 𝑝̂𝑗 events in the kth group, where mj is the number of observations in jth covariate pattern, and 𝑝̂𝑗 is the estimated probability ck ∑ for jth covariate pattern. Similarly, O0k m Y is the observed number of nonevents (Yj 0) in the kth group, = 𝑗=1( 𝑗 − 𝑗 ) = ck and Ê0k = m (1 − ) is the estimated expected number of nonevents in the kth group. Since g = 10 and g = 20 are 𝑗=1 𝑗 𝑝̂𝑗 ∑ commonly suggested in practice, we use these 2 settings to perform HL tests in Section 3. With a bit of algebra, it can be ∑ shown that g ′ 2 (O1k − nk𝑝̄k) HL = n′ 1 , k=1 k𝑝̄k( − 𝑝̄k) ∑ ′ where nk is the number of observations in the kth group and 𝑝̄k is the average estimated probability in the kth group. Whenthenumberofgroupsg equals to the sample size n, the formula of HL can be further rewritten as n 2 (Yi − 𝑝̂i) HL = 1 , i=1 𝑝̂i( − 𝑝̂i) ∑ which suggests the formula for another GOF measure: the sum of squared standardized residuals. 1.1.2 Stukel statistic To perform Stukel GOF test,4 1 or 2 additional superfluous covariates are included in the standard model (1), and this ⊤ new model is tested versus the original one. For example, let 𝑝̂i = F(xi 𝛽̂) be the estimated original model, where 𝛽̂ is 2 2 ⊤ the vector of estimated coefficients. Let za = gi I(gi ≥ 0) and zb = gi I(gi < 0) be the 2 covariates, where gi = xi 𝛽̂.After ⊤ including za and zb into the model, it becomes 𝑝̂i = F(xi 𝛽̂ + za𝛽̂a + zb𝛽̂b). Then, the Wald test or the likelihood ratio test can be performed to test the null hypothesis that both 𝛽a and 𝛽b are equal to 0. 1.1.3 Unweighted sum of squares statistic When some of the expected cell frequencies in the 2 × n contingency table are too small, the weighted chi-square test for proportions cannot be used. In this case, Copas5 modified the chi-square test as n 2 USS = (Yi − 𝑝̂i) , i=1 ∑ where 𝑝̂i is defined as before and n is the sample size. Under the null hypothesis, this statistic will have an asymptotic normal distribution when standardized by its mean and the square root of its variance. 1.1.4 Information matrix statistic In 1982, White6 introduced a general test for model misspecification. The basic idea of this test is to compare 2 different estimates of the covariance matrix. If these 2 estimates are the same, there is no lack of fit. The formula for information matrix statistic is n 𝑝 2 IM Yi i 1 2 i x = ( − 𝑝̂ )( − 𝑝̂ ) i𝑗 , i=1 𝑗=0 ∑∑ 6 where xij is the jth predictor for the ith observation. As shown by White, IM has an approximate chi-square distribution with p + 1 degree of freedom under the null hypothesis of the correct model. 2 THE SUM OF STANDARDIZED RESIDUALS STATISTIC As we mentioned in Section 1, the sum of squared standardized residuals is also a GOF measure in binary response model n 2 n n (Yi − 𝑝̂i) 1 − 2𝑝̂i 𝑝̂i Tn = = Yi + = Wi(𝑝̂i). i(1 − i) i(1 − i) 1 − i i=1 𝑝̂ 𝑝̂ i=1 (𝑝̂ 𝑝̂ 𝑝̂ ) i=1 ∑ ∑ ∑ 105 CHEN ET AL. 3 This statistic was proved to be asymptotic following the standard normal distribution when properly standardized.7 However, the drawback of Tn statistic is that its variance may be easily dominated by the variances of its individual com- ponents Wi(𝑝̂i) corresponding to extreme numerical values of pi. Additionally, Wi(𝑝̂i) become approximately constant around pi = 1∕2, which sometimes may be undesirable and lead to loss of efficiency. One intuitive way to address these 2 2 issues is to assign a variance-stabilizing weight to Wi(𝑝̂i).SinceEWi(pi)=1andVarWi(pi)=(1 − 2pi) ∕(pi(1 − pi)),the (approximately) centered and variance-stabilized new statistic is n Yi − 𝑝̂i Cn = . i=1 𝑝̂i(1 − 𝑝̂i) ∑ We will analyze the asymptotic distribution of this statistic√ under the following assumptions. 2.1 Regularity assumptions 7 Following Windmeijer, we assume that (1) holds for some 𝛽 = 𝛽0 as well as the following: ′ 1. Both the first-order derivative f and second-order derivative f of F are continuous. There exists a positive constant M1 < ∞ such that 0 < F(z) < 1andf(z) > 0forall z ≤ M1. k 2. 𝛽0 ∈ , the parameter space that is an open subset of the Euclidean space R . ⊤ 3. There exists a positive constant M2 < ∞ such that | x|i ≤ M2, ∀i ∈ N and lim ixix is a finite nonsingular matrix. n→∞ i Also, the empirical distribution of x converges to a nondegenerate distribution function. i ∑ || || 8 Under assumptions 1 to 3, the following 2 results about MLE 𝛽̂n may be established (see, eg, Amemiya ). Here and d elsewhere in the paper, we denote convergence in distribution by →. Asymptotic distribution of MLE We have for n → ∞ 1 d −1 n 2 (𝛽̂n − 𝛽0) → N(0, Ω ), (2) where 𝛽̂n is the MLE of 𝛽0 and Ω is defined as n 2 −1 𝑓i0 ⊤ Ω = lim n xix , n→∞ 1 i i=1 𝑝i0( − 𝑝i0) ∑ ⊤ ⊤ with 𝑝i0 = F(xi 𝛽0) and 𝑓i0 = 𝑓(xi 𝛽0). Applying the Taylor expansion argument, one can also show the following. Score representation of MLE 𝛽̂n satisfies 2 −1 𝜕log L 𝜕logL ̂ ∗ (3) 𝛽n − 𝛽0 =− T 𝛽n 𝛽0 , ( 𝜕𝛽𝜕𝛽 ) 𝜕𝛽 ∗ where 𝛽n lies on the segment between 𝛽̂n and 𝛽0 and | | n logL = Yilog𝑝i +(1 − Yi)log(1 − 𝑝i) i=1 ∑ ( ) is the log-likelihood of the binary response model. As it turns out, these 2 results allow for a simple asymptotic approximation of Cn. 2.2 Asymptotic distribution of Cn We describe the asymptotic distribution of Cn in the following theorem, which we prove in the Appendix. Theorem 1. Under assumptions 1 to 3, it can be shown that as n → ∞ 1 − n 2 Cn d → N(0, 1), 𝜎n where 2 ⊤ −1 𝜎n = 1 − vn Ω vn, 106 4 CHEN ET AL. with n −1 𝑓i0xi vn = n . (4) i=1 𝑝i0(1 − 𝑝i0) ∑ √ 3 NUMERICAL EXAMPLES On the basis of the theorem, a new GOF test can be defined. Let 𝜎̂n be the estimate of 𝜎n where we replace 𝛽0 with 𝛽̂n. The test statistic is then Cn C̃ n = , (5) n𝜎̂n which, according to Theorem 1, asymptotically follows the standard normal distribution. To show the application of this √ result and evaluate the power of Cn along with other GOF statistics, we provide several numerical examples under differ- ent scenarios. Statistics considered here include Hosmer-Lemeshow3 (HL)withg = 10 and g = 20, Stukel,4 unweighted 5 6 sum of squares (USS), information matrix (IM), and our new proposed statistic Cn. The tests are all performed at the 𝛼 level of .05. Note that while Cn leads to 2-sided test, its competitors considered here are mostly 1-sided. As the simulated covariates might not be biologically meaningful, we consider the covariates based on values from real datasets. In other words, we only generate the outcome variable Yi. In this setting, the power to detect the omission of a quadratic term or an interaction term is evaluated. For each simulation, the number of replications is 10 000. 3.1 Linear vs quadratic Covariates used in this section are derived from an epidemiological study conducted among runners in the 2002 Boston Marathon.9 The goal of this study was to identify potential risks associated with hyponatremia, which is defined as a serum sodium concentration level of 135 mmol/L or lower. Details about the selected covariates and the outcome variable are provided in Table 1. After excluding participants with missing values in selected covariates or the outcome variable, we are left with n = 445. The models that we used to generate Yi are as follows: exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui) 𝑝i = , (Model 1.0) 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui) 2 exp(−5 + 0 7si + 0 7wi + 0 013ri + 0 9ui + 0 1wi ) . . . . . (Model 1.1) 𝑝i = 2 , 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.1wi ) 2 exp(−5 + 0 7si + 0 7wi + 0 013ri + 0 9ui + 0 2wi ) . . . . . (Model 1.2) 𝑝i = 2 . 1 + exp(−5 + 0.7si + 0.7wi + 0.013ri + 0.9ui + 0.2wi ) Model 1.0 contains no quadratic term, and models 1.1 and 1.2 contain quadratic terms with different coefficients, which are to emphasize or deemphasize the effect of the quadratic term. Results comparing the H0 rejection proportion among different tests are showed in Table 2. In Table 2, we see that under the null (true) model, nearly all test statistics reject at about the 𝛼 level of .05 except Stukel and HL(g = 20). Table 2 also shows that under model 1.1, which added a quadratic term with a coefficient of 0.1, rejection proportions for all statistics increase. It is worth mentioning that Cn has the highest power here. Under model 1.2, as the value of the coefficient increases, the power of detecting such departure becomes much greater. Indeed the tests based on Cn, Stukel, and IM all have the power exceeding 90%,withCn again attaining the highest power. TABLE 1 Selected variables description in the Boston Marathon data Symbol Codes/Values Variable description y 0 = ≤135 mmol/L, 1 = >135 mmol/L Serum sodium concentration at race completion s 0 = female, 1 = male Sex w kg Weight change during race r hours Race duration u 1 = >3L,0= ≤3 L Fluid consumption during the race 107 CHEN ET AL. 5 TABLE 2 Proportion H0 rejected at the 𝛼 = .05 using sample size of 445 with 10 000 replications in quadratic models using the Boston Marathon data Model Test statistics Proportion H0 rejected Under H0: Model 1.0 Cn 0.042 HL(g = 10) 0.059 HL(g = 20) 0.071 USS 0.058 Stukel 0.072 IM 0.051 Under Ha: Model 1.1 Cn 0.337 HL(g = 10) 0.224 HL(g = 20) 0.300 USS 0.306 Stukel 0.287 IM 0.234 Under Ha: Model 1.2 Cn 0.985 HL(g = 10) 0.847 HL(g = 20) 0.921 USS 0.977 Stukel 0.976 IM 0.964 Abbreviations: HL, Hosmer-Lemeshow; IM, information matrix; USS, unweighted sum of squares. TABLE 3 Selected variables description in Cleveland data Symbol Codes/Values Variable description y 0 = absence, 1 = presence Heart disease diagnosis s 0 = female, 1 = male Sex t mmHg, recorded at baseline Resting blood pressure 3.2 Linear vs interaction An additional set of covariates is used to measure power against superficial interaction and comes from the Cleveland dataset, which is a subset of a large heart disease dataset.10 Only 2 covariates, denoted below as s and t, are chosen to be included in our simulation on the basis of their significant relationship with the response variable. After the exclusion of observations with missing values in any of the 3 variables, a total of 303 observations is left in the dataset. Table 3 shows a description of these 3 variables. We used 2 models to generate Yi: exp(−4 + 1.4si + 0.02ti) 𝑝i = , (Model 2.0) 1 + exp(−4 + 1.4si + 0.02ti) exp(−1 + 4.5si + 0.03ti − 0.065si × ti) 𝑝i = . (Model 2.1) 1 + exp(−1 + 4.5si + 0.03ti − 0.065si × ti) Model 2.0 is the null model that contains 2 covariates s and t. Model 2.1 is the alternative model that contains not only s and t but also an interaction term between the 2 covariates. Table 4 shows the percent of time H0 was rejected by each of the 6 tests except Hosmer-Lemeshow with group = 20 because of the limited number of covariate patterns in this situation. We see in Table 4 that, as expected, the proportions of rejecting the null hypothesis are still close to 0.05 when the fitted model is correct (except for HL). Also, when an interaction term is added to the true model, rejection proportions are greater than those under null. In this setting, while both HL with a group of size 10 and IM show poor power, Cn performs noticeably better than the others with the power of 65.3%. However, it is unclear whether this pattern continues to hold when the true model departs further from the fitted model. 108 6 CHEN ET AL. TABLE 4 Proportion H0 rejected at the 𝛼 = .05 using sample size of 303 with 10 000 replications in interaction models using the Cleveland data Model Test statistics Proportion H0 rejected Under H0: Model 2.0 Cn 0.048 HL(g = 10) 0.033 USS 0.048 Stukel 0.056 IM 0.043 Under Ha: Model 2.1 Cn 0.653 HL(g = 10) 0.263 USS 0.554 Stukel 0.557 IM 0.446 Abbreviations: HL, Hosmer-Lemeshow; IM, information matrix; USS, unweighted sum of squares. TABLE 5 Selected variables description in NHANES data Symbol Codes/Values Variable description y 0 = PEPMNK1R ≤ 140, 1 = PEPMNK1R > 140 High blood pressure s 0 = female, 1 = male Sex c mg/100 mL Serum cholesterol TABLE 6 Proportion H0 rejected at the 𝛼 = .05 using sample size of 373 with 10 000 replications in interaction models using the NHANES data Model Test statistics Proportion H0 rejected Under H0: Model 3.0 Cn 0.052 HL(g = 10) 0.038 HL(g = 20) 0.039 USS 0.051 Stukel 0.055 IM 0.050 Under Ha: Model 3.1 Cn 0.092 HL(g = 10) 0.049 HL(g = 20) 0.050 USS 0.074 Stukel 0.094 IM 0.065 Under Ha: Model 3.2 Cn 0.326 HL(g = 10) 0.095 HL(g = 20) 0.070 USS 0.336 Stukel 0.297 IM 0.226 Abbreviations: HL, Hosmer-Lemeshow; IM, information matrix; NHANES, National Health and Nutrition Examination Survey; USS, unweighted sum of squares. To find out whether further departure changes the performance, another set of covariates is used. It comes from the National Health and Nutrition Survey and contains 17 030 observations and 16 variables. Two independent variables, s and c, are chosen to run the simulation. They are simultaneously significantly associated with the dependent variable y. 109 CHEN ET AL. 7 We randomly select 373 observations (to keep the dataset size comparable with that of the previous examples) with no missing value in s, c,andy. Table 5 gives a description of these 3 variables. We used 3 models to generate Yi: exp(−1.5 + 0.005ci + 1.24si) 𝑝i = , (Model 3.0) 1 + exp(−1.5 + 0.005ci + 1.24si) exp(−0.6 + 0.0007ci + 0.5si + 0.003ci × si) 𝑝i = , (Model 3.1) 1 + exp(−0.6 + 0.0007ci + 0.5si + 0.003ci × si) exp(−0.6 + 0.0007ci + 0.5si + 0.01ci × si) 𝑝i = . (Model 3.2) 1 + exp(−0.6 + 0.0007ci + 0.5si + 0.01ci × si) Model 3.0 is the null model containing only the linear term, and models 3.1 and 3.2 are 2 alternative models designed to assess the power of detecting the omission of an interaction component. The variation in the coefficients of the last 2 models is introduced to emphasize and deemphasize the effect of omitting an interaction term. Table 6 shows that the percent of time H0 was rejected by each of the 6 tests using the National Health and Nutrition Examination Survey (NHANES) data. In Table 6, we see that under null, proportions of H0 being rejected among all statistics remain close to the nominal level of 0.05. Except for 2 Hosmer-Lemeshow tests, the power of the other tests increases to between 7% and 10% when adding an interaction term with the coefficient of 0.003. As the coefficient increases to 0.01, the power of almost all statistics increases to around 30%. Although no longer the top performer in either case, note that only Cn attains high power in both situations. 4 MODEL SELECTION EXAMPLE As our final example, we present an application of the SSRs statistic Cn to the model selection problem by comparing its fit under different link functions. The dataset was taken from the National Cancer Institute's Health Information National Trends Survey (HINTS 2007), which is a national effort to monitor efforts in public health knowledge dissemination among general public and its effects on cancer prevention and monitoring. More detailed information on the HINTS 2007 survey can be found elsewhere.11 For our current example, a subset of HINTS data was selected with 3 variables of interest described in Table 7. Here, the variable cancer is the outcome of interest, with age and race being 2 explanatory variables. Three samples (each of size 100) were randomly drawn to assess the performance of Cn for small-to-moderate sample sizes. For each sample, the GOF test using Cn under 3 different models represented by the link functions—logit, probit, and complementary log-log—was performed, and the asymptotic P values were used for data-based models discrimination. Table 8 presents the respective values of the statistic Cn and the corresponding P values under each link. As seen from the table, the test statistic Cn gives evidence against the probit link but not against the remaining 2 links (which very TABLE 7 Selected variables description in HINTS data Symbol Codes/Values Variable description cancer 0 = absence, 1 = presence Previous cancer diagnosis age years Age race 1 = white, 2 = black, 3 = unknown Race Abbreviation: HINTS, Health Information National Trends Survey. TABLE 8 The approximate CnP values under each model using the HINTS data Sample 1 Sample 2 Sample 3 Link Function Cn P Value Cn P Value Cn P Value Logit 1.08 .28 −0.498 .62 −1.06 .29 Probit 3.78 .00016 −2.16 .03 −4.07 .00005 Cloglog 1.08 .28 −0.474 .63 −1.16 .25 Abbreviation: HINTS, Health Information National Trends Survey. 110 8 CHEN ET AL. consistently gave similar values of Cn across all 3 samples). Consequently, our analysis indicates that the model with the probit link is not supported by the data, whereas the other 2 models possibly are. 5 SUMMARY In this work, we have proposed a new statistic for GOF in a binary response model. This statistic denoted here Cn is based on the SSRs. Under the regularity assumptions on the model, we have shown that Cn asymptotically follows a normal distribution. This result can be used to perform asymptotic P value calculations for GOF. Under the null model, we see that H0 is rejected at the rate close to the nominal level 5% as long as the sample size is not too small. Under the alternative, the overall performance of Cn is satisfactory across all of the scenarios we proposed here. When the binary response model contains multiple covariates, the overall performance of Cn dominates the performance of alternative GOF statistics considered. On the basis of our results, Cn seems to be a well-powered and intuitive GOF statistic, which is also simple to use. Although in the paper we has focused exclusively on the binary response problem, it appears that the Cn-based GOF tests could be used beyond binary data in very much the same way. Two natural extensions of the model described here would be the multinomial and continuous responses in the generalized linear model setting. This would require a generalization of our Theorem 1, and we leave this problem for possible future investigation. ACKNOWLEDGEMENTS The research was partially supported by the Mathematical Biosciences Institute, which receives its funding from the National Science Foundation under grant DMS 1440386. The authors would like to acknowledge helpful comments from the 2 referees on an earlier version of the manuscript. ORCID Grzegorz A. Rempala http://orcid.org/0000-0002-6307-4555 REFERENCES 1. Salomon CJ, Ridder G. The logit model in economics. Statistica Neerlandica. 1988;42(4):297-314. 2. Hosmer David W, Trina H, Le CS, Stanley L, et al. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965-980. 3. Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat Theory Methods. 1980;9(10):1043-1069. 4. Stukel Thérèse A. Generalized logistic models. J Am Stat Assoc. 1988;83(402):426-431. 5. Copas JB. Unweighted sum of squares test for proportions. Appl Stat. 1989;38:71-80. 6. Halbert W. Maximum likelihood estimation of misspecified models. Econometrica: J Ecol Soc. 1982;50(1):1-25. 7. Windmeijer FAG. The asymptotic distribution of the sum of weighted squared residuals in binary choice models. Statistica Neerlandica. 1990;44(2):69-78. 8. Amemiya T. Advanced Econometrics. Boston, MA: Harvard University Press; 1985. 9. Almond Christopher SD, Shin Andrew Y, Fortescue Elizabeth B, et al. Hyponatremia among runners in the Boston Marathon. NEnglJ Med. 2005;352(15):1550-1556. 10. Lichman M. UCI Machine Learning Repository; 2013. 11. Cantor D, Coa K, Crystal-Mansour S, Davis T, Dipko S, Sigman R. Health Information National Trends Survey (HINTS) 2007: Final Report. 2009. 12. Shao J. Mathematical Statistics. New York, USA: Springer; 2003. Howtocitethisarticle: Chen L, Sui Y, Song C, Rempala GA. The sum of standardized residuals: Goodness-of-fit test for binary response models. Statistics in Medicine. 2018;1–10. https://doi.org/10.1002/sim.7644 111 CHEN ET AL. 9 APPENDIX : ASYMPTOTIC DISTRIBUTION OF CN Define n (Yi − 𝑝̂i) Sn(𝛽̂n)= . i=1 𝑝̂i(1 − 𝑝̂i) 1 ∑ − Local expansion of n 2 Sn(𝛽̂n) around 𝛽0 gives, as n → ∞, √ 1 1 Sn( ) 1 − 2 2 −1 𝜕 𝛽 2 n Sn(𝛽̂n)=n Sn(𝛽0)+n 𝛽0 n (𝛽̂n − 𝛽0)+o𝑝(1), 𝜕𝛽⊤ n (Yi−𝑝i0) where Sn(𝛽0)= i=1 . Observe that | 𝑝i0(1−𝑝i0) ∑ √ n 3 −1 𝜕Sn(𝛽) −1 1 1 − n = n i0xi − − (Yi − i0)( i0(1 − i0)) 2 (1 − 2 i0) 𝛽0 𝑓 2 𝑝 𝑝 𝑝 𝑝 𝜕𝛽 i=1 ( 𝑝i0(1 − 𝑝i0) ) ∑n n 3 | −1 1 √ − −1 𝑓i0xi =−n i0xi(Yi − i0)( i0(1 − i0)) 2 (1 − 2 i0)−n 2𝑓 𝑝 𝑝 𝑝 𝑝 i=1 i=1 𝑝i0(1 − 𝑝i0) ∑ ∑ =∶ Un − vn, √ where n 3 −1 1 − Un =−n i0xi(Yi − i0)( i0(1 − i0)) 2 (1 − 2 i0) 2𝑓 𝑝 𝑝 𝑝 𝑝 , i=1 n∑ −1 𝑓i0xi vn = n . i=1 𝑝i0(1 − 𝑝i0) ∑ 2 We note that plimUn 0, since EUn 0, and for each component of vector Un,sayUj,n,wehaveEU → 0asn → = = √ 𝑗,n − ∞ in view of n 2 2 1 (1 − 2𝑝i) EU = i0x i 𝑗,n 4n2 1 2 𝑓 𝑗, i=1 (𝑝i( − 𝑝i)) ∑n 1 ≤ i0x i 4n2A 𝑓 𝑗, i=1 ∑ n x i D i=1 𝑗, ≤ , 4nA ∑ n ⎛ ⎞ n x ⎜ ⎟ where 0 A p , f D ,and i=1 𝑗,i converges because of our assumptions 1 to 3. Furthermore, v is bounded, since < ≤ i i0 ≤ < ∞ n ⎜ ⎟ n ∑ ⎝ ⎠ n n n 2 2 −1 𝑓i0xi 2 −1 𝑓i0xi 2 −1 𝑓i0 2 vn = n ≤ n = n xi i0(1 − i0) i=1 𝑝i0(1 − 𝑝i0) i=1 𝑝i0(1 − 𝑝i0) i=1 𝑝 𝑝 ∑ ∑ ∑n 2 || || || √ || ||√ || −1 𝑓i0 || ||⊤ = n tr xix 1 i , i=1 𝑝i0( − 𝑝i0) ∑ n 1 −1 ⊤ − 2 and n tr i=1 xixi converges under our assumptions. Therefore, it follows that n Sn(𝛽̂n) has the same limiting distribution as ∑ 1 1 − ⊤ n 2 Sn(𝛽0)−vn n 2 (𝛽̂n − 𝛽0). 2 𝜕log L −1 𝜕logL Using (3) in Section 2.1, we know that ̂n − 0 =−( ∗ ) , and therefore, 𝛽 𝛽 𝜕𝛽𝜕𝛽⊤ 𝛽n 𝜕𝛽 𝛽0 1 1 logL ⊤ 2 ⊤ − 2 ∗ −1 𝜕 vn n (𝛽̂n − 𝛽0)=vn| n (An) | 𝛽0 , 𝜕𝛽 log2L 1 d ∗ −1 𝜕 2 −1 | −1 ∗ where An =−n ∗ .Sincevn is bounded, n ( ̂n − 0) →− N(0 Ω ) by (2) and lim Ω An = I,wehave 𝜕𝛽𝜕𝛽⊤ 𝛽n 𝛽 𝛽 , 𝑝 1 ⊤ −1 ∗ | 𝑝lim vn (Ω An − I)n 2 (𝛽̂n − 𝛽0)=0. 112 10 CHEN ET AL. 1 12 − Thus, by (3) and Theorem 1.10 of Shao, n 2 Sn(𝛽̂n) has the same limiting distribution as 1 1 logL − 2 ⊤ − 2 −1 𝜕 n Sn(𝛽0)−vn n Ω 𝛽0 . 𝜕𝛽 Since 𝜕logL n Yi−𝑝i0 x ,then 𝛽0 = i=1 𝑓i0 i | 𝜕𝛽 𝑝i0(1−𝑝i0) ∑ 1 1 logL − 2 ⊤ − 2 −1 𝜕 | n Sn(𝛽0)−vn n Ω 𝛽0 𝜕𝛽 n n 1 Yi − i0 1 Yi − i0 − 2 𝑝 ⊤ |− 2 −1 𝑝 = n − vn n Ω 𝑓i0xi i0(1 − i0) i=1 𝑝i0(1 − 𝑝i0) i=1 𝑝 𝑝 ∑ ∑ n 1 1 ⊤ − − √(Yi − 𝑝i0)( 𝑝i0(1 − 𝑝i0)−vn Ω 𝑓i0xi) = n 2 1 . i=1 √ 𝑝i0( − 𝑝i0) ∑ Define ⊤ −1 (Yi − 𝑝i0)( 𝑝i0(1 − 𝑝i0)−vn Ω 𝑓i0xi) Zni = √ 𝑝i0(1 − 𝑝i0) ⊤ −1 Yi − 𝑝i0 vn Ω 𝑓i0xi = 1 − . 𝑝i0(1 − 𝑝i0) ( 𝑝i0(1 − 𝑝i0)) Note that Zn1, … , Znn are independent random√ variables. The√ moments of Zni are EZni = 0 2 ⊤ −1 2 vn Ω 𝑓i0xi 2 EZni = 1 − =∶ 𝜎ni ( 𝑝i0(1 − 𝑝i0)) 3 1 2 2 2 ⊤ −1 3 ( − 𝑝i0 + √𝑝i0) vn Ω 𝑓i0xi E Zni = 1 − =∶ mni. 𝑝i0(1 − 𝑝i0) | 𝑝i0(1 − 𝑝i0)| | | | | √ n m | √ | By the Lyapunov central limit theorem, since i=1 ni| 0inviewofassumptions1to3accordingto| lim n |3 = | n→∞ 2 |2 | ( ∑i=1 𝜎ni) n ∑ m n max mni max mni ni i 1 i i=1 O n−1∕2 n 3 ≤ 3 = 3 = ( ), ∑ 2 3 (min ni) ( ) 2 n 2 (min ni) n i 𝜎 i=1𝜎ni i 𝜎 the sum of weighted residuals converges∑ in distribution to the standard√ normal random variable. Therefore, as n → ∞, ⊤ −1 Y − v Ω 𝑓i0xi n i 𝑝i0 1 − n (1− ) (1− ) d 𝑝i0 𝑝i0 𝑝i0 𝑝i0 →N 0 1 ( )1 − ( , ). √ n √ i=1 2 2 v⊤Ω−1 x ∑ 1 − n 𝑓i0 i 𝑝i0(1−𝑝i0) ( i=1 ) ∑( √ ) 2 n 2 Finally, by (2) and (4), the square of the denominator above, say, 𝜎n = i=1 𝜎ni, can be rewritten as follows: n 2 ⊤ −1 ∑ 1 vn Ω 𝑓i0xi 2 = 1 − 𝜎n n i=1 ( 𝑝i0(1 − 𝑝i0)) ∑ n n x 2 ⊤ −1√1 𝑓i0 i ⊤ −1 1 𝑓i0 ⊤ −1 = 1 − 2vn Ω + vn Ω xixi Ω vn n n i0(1 − i0) i=1 𝑝i0(1 − 𝑝i0) i=1 (𝑝 𝑝 ) ∑ ∑ ⊤ −1 = 1 − vn Ω vn + o(√1). This completes the proof of Theorem 1. The asymptotic normality of the statistic C̃ n defined in (5) follows now from the 2 2 fact that 𝜎n∕𝜎̂n → 1 in probability due to the consistency of the MLE estimate 𝛽̂n, the regularity condition 3, and the Toeplitz lemma (see, eg, chapter 1 of Shao12). 113