Hierarchical Modeling with SAS GLIMMIX Jian Dai, Zhongmin Li, David Rocke University of California, Davis, CA

ABSTRACT

Data often have hierarchical or clustered structures, such as patients clustered within hospitals or students nested within schools. Hierarchical models are statistical models that are used to analyze hierarchical or multilevel . SAS GLIMMIX procedure is a new and highly useful tool for hierarchical modeling with discrete responses. This paper is focused on hierarchical logistic regression modeling with GLIMMIX. We present several applications of these models and show how to use GLIMMIX to fit the models and test hypotheses. We illustrate the applications using a sample data from a multi-institution database on coronary artery bypass grafting surgeries developed by the California Office of Statewide Health Planning and Development.

INTRODUCTION

In many applications, data have hierarchical or clustered structures, such as medical and health services research where patients are clustered within hospitals, or educational studies where students are nested within schools. These studies often involve the analysis of data with complex patterns of variability, such as multilevel, nested sources of variability. For example, in the quality analysis of healthcare providers, there is variability between the providers as well as between patients who are nested within the providers. Hierarchical models are statistical models that can be used to analyze nested sources of variability in hierarchical data, taking account of the variability associated with each level of the hierarchy. These models have also been refereed to as multilevel models, mixed models, random coefficient models, and component models (Breslow and Clayton, 1993; Longford, 1993; Snijders and Bosker, 1999; Hox, 2002; Goldstein, 2003). In applications, the outcome variable is often binary. For example, the outcome of a medical treatment might be a success or failure; the patient might have survived or died after a surgery; a tissue sample might be normal or cancerous. Binary outcomes lead to a generalized with the logic link, which is the logistic regression model. In this paper we are focused on hierarchical logistic regression models, which can be fitted using the new SAS procedure GLIMMIX (SAS Institute, 2005). Proc GLIMMIX is developed based on the GLIMMIX macro (Little et al., 1996) and provides highly useful tools for fitting generalized linear mixed models, of which the hierarchical logistic model is a special case.

We will show how to use GLIMMIX to fit hierarchical logistic models. We will discuss that hierarchical models and mixed models are equivalent, two ways to express the same relationships. We will introduce random intercept models and demonstrate their usefulness using examples. We use GLIMMIX to fit the models and test hypotheses. The focus of the paper is on two-level hierarchical models, but it is straightforward to fit models with higher hierarchies. The rest of the paper is organized as follows. We first motivate the application with an example in pubic health study; then develop the models and describe the code for fitting the models. We then describe the data and present the results, and finally conclude the study.

MOTIVATION

Health care quality analysis is an important area in public health study. One way to assess the quality of care is to analyze between and within provider variations in healthcare outcome. This typically involves analysis of hierarchical data at different levels, for example, the patient-level, the physician-level, and the hospital-level. In the simplest case, two levels of analysis are needed: the patient level and a single provider level. The patient level analysis is needed because the outcome measures are meaningful only when adequately adjusted for patient risks. At the provider level, the risk due to the provider is analyzed; the differences between providers are compared and evaluated. It is clear that in such applications the data structures are hierarchical and the analyses are multi-level (Goldstein and Spiegelhalter, 1996; DeLong, 1997; Shahian et al., 2001).

There are different modeling approaches for such analysis. One approach is to take steps for different levels of analyses: model the data at the lowest level and then aggregate the modeling results and perform analysis on higher level units. For example, one may begin with patient-level data, estimating a patient-level risk model, computing expected mortality of the provider by summing up the risks of its patients, and then compare the expected mortality to the observed mortality of the provider. An alternative and less approximate approach would be to model the hierarchical data in one step. The hierarchical models take account of the variability at each level of the hierarchy and

1 thus allow the provider effects to be analyzed within the models. The hierarchical models also have other advantages (Normand et al., 1997; Shahian et al., 2001). For example, they can account for clustering of observations, such as patients nested within hospitals, while the traditional regression models assume independence of observations. Hierarchical models assume that higher level units are drawn from a population of units and produce posterior or predicted estimates of unit effects. These estimates are “shrunken” estimates which have the useful property of moving higher level unit estimates towards the population (regression to the mean) and increasing the accuracy of prediction (Goldstein and Spiegelhalter, 1996).

One area in healthcare that has received much attention is quality assessment for coronary bypass grafting (CABG) surgeries. Information on how well hospitals perform this surgery is critical for hospital quality improvement efforts and assisting patients and their families in making informed decisions about where to receive the best care. For this purpose, many states collect data and publish reports on CABG surgeries. In this paper, we use a sample data from the California CABG 2000-2002 database (Parker et al., 2005) to illustrate the application of hierarchical logistic regression models. We demonstrate the application with two examples: (1) assessing differences in hospital effects on in-hospital mortality; and (2) evaluating the effects of hospital teaching status on the mortality. The purpose of the application is for illustrating the methods, not for policy analysis.

MODELS

We begin with the ordinary logistic regression model, which is a single level model but provides a starting point for developing multilevel models for binary outcomes. We then present the random intercept models which have many applications in public health and other studies. For simplicity of presentation, we consider two-level models, for example, models accounting for patient-level and hospital-level effects. The two-level data structure is shown in Figure 1 below:

Figure 1. Data Structure for a Two-Level Hierarchical Model

In this data structure, level-1 is the patient level and level-2 is the hospital level. Within each level-2 unit there are nj patients in the j’th hospital. We further simplify the presentation by assuming there is a single patient-level predictor x (e.g. a risk index) and one hospital-level factor z (e.g. size or type of hospital). The models can be easily generalized to handle more complicated data structures (Hox, 2002; Goldstein, 2003; Demidenko, 2004).

ORDINARY LOGISTIC REGRESSION MODEL

Suppose that y is a binary outcome variable (e.g. the patient survived or died after a surgery) and follows the Bernoulli distribution, y ~ Bin(1,π ) and x is a patient-level predictor. Then, the ordinary logistic regression model (Hosmer and Lemeshow, 2000) is

yij = π ij + eij , (1) ⎛ π ⎞ logit(π ) = log⎜ ij ⎟ = α + βx ij ⎜ ⎟ ij ⎝1− π ij ⎠

where i = 1,...,I j is the patient level indicator, j = 1,..., J is the hospital level indicator, andπ ij is the probability of death for patient i in hospital j, conditional on the risk factor x. The logit model assumes that patient level random

2 2 errors eij are independent with moments E(eij ) = 0 and Var(eij ) = σ e = π ij (1 − π ij ) . The logit model has a linear function at the logit (log odds) scale. Equation (1) implies that the probability function is

exp(α + βxij ) π ij = (2) 1+ exp(α + βxij )

We have used two subscripts (i,j) to reflect the fact that the data have two levels and the patients are nested within hospitals. However, this model is a single-level model because it doesn’t contain hospital level effects. Nor does it account for the variation between hospitals and the clustering of patients within hospitals.

HIERARCHCIAL LOGISTIC REGRESSION MODEL

There are several ways to extend the single-level model to multilevel analysis. A simple way to account for effects of higher-level units is to add design variables (dummy variables) to Equation (1) so that each higher-level unit (in this case, each hospital) has its own intercept in the model. These hospital intercepts (subject-specific intercepts) are used to measure the differences between hospitals,

logit(π ij ) = α j + βxij (3)

Equation (3) is such a model specified at the logit scale, where each hospital j has its own intercept α j . The intercepts can be specified as either fixed effects or random effects (Demidenko, 2004). The use of fixed intercepts, however, leads to increasing the number of additional parameters equal to the number of higher-level units minus 1 (J-1). Thus, if the number of hospitals is large, one faces the problem of a large number of nuisance parameters in the model and can have very poor estimation results. A more sophisticated approach is to treat the hospital intercepts, α j (j=1,…,J), as a with specified , which leads to a random intercept model and more conservative estimates of hospital effects:

logit(π ij ) = α j + βxij (4) α j = α + u j

In this model, the hospital effects are measured by the random intercepts α j (j=1,…J), a linear combination of a grand mean (α ) and a deviation ( u j ) from that mean. In GLIMMIX the random variable u j is assumed to be 2 normally distributed u j ~ N(0,σ u ) and independent of the patient level random errors ( eij ). The hospital intercepts measure the differences between the hospitals, controlling for other effects in the model such as patient risks. The model in (4) is a hierarchical model with two levels of hierarchy. At level 1, we express the outcome as the sum of an intercept for the patient’s hospital and the patient’s risk factor. At level 2, we specify the hospital level intercepts as the sum of an overall mean and the random deviations from that mean. The equations for the two levels can be easily combined to form one equation,

logit(π ij ) = α + u j + βxij (5)

Equation (5) is a mixed model because it has both fixed effects (α , β ) and random effects ( u j ). It is a logistic mixed model, because the link function is logit, and thus, a member of the family of generalized linear mixed models. One way to fit this model with GLIMMIX is as follows proc glimmix; class hospital; model y = x / dist=binary link=logit ddfm=bw solution; random intercept / subject=hospital solution; run;

In the example, the variable HOSPITAL is a classification variable that identifies the hospital from which the patient received surgery; thus it is on the CLASS statement. The MODEL statement is for specifying the response variable, the fixed effects and the options for the modeling. In the example, y is the response variable and x is the fixed effects

3 for patient-level risks. Using the options DIST=BINARY and LINK=LOGIT, we specify that the response variable is binary distributed and the link function is logit. The option /DDFFM=BW asks GLIMMIX to use the method ”between/within” for computing the denominator degrees of freedom for tests of fixed effects. The option SOLUTION requests to print the solution for fixed effects. We use the RANDOM statement to specify INTERCEPT as the random effects. The hierarchical structure is specified through the SUBJECT option, indicating that the patients are nested within hospitals. We request to print the estimates of random effects by using the SOLUTION option. In generalized linear mixed models, random effects shrinkage may cause discrepancy between observed and predicted residual . To help overcome this discrepancy, one may use an extra RANDOM statement with the _RESIDUAL_ keyword.

APPLICATION FOR HOSPITAL PROFILING

Random intercept models have many applications. We illustrate it with an application for hospital profiling. Suppose that we have a sample of hospitals from a population of hospitals, a dichotomous variable y indicating the outcome of a surgery, and some patient risk measure x. We can use the random intercept model defined in equation (5) to estimate the hospital effects on mortality, adjusting for individual patients’ risks, and within the model, evaluate and compare the performance of the hospitals. We can use GLIMMIX to fit the model and obtain estimates of the hospital effects, u j , and the hospital intercepts αˆ j = αˆ + uˆ j . We can obtain the for each hospital by exponentiating the estimated value of the hospital effect, exp(uˆ j ) , which is a measure of risk due to the hospital relative to the average of all (DeLong et al., 1997). For convenience, we call this hospital-specific odds ratio as HRISK. Similar to the use of O/E ratio, if HRISK is greater than one, the risk of the hospital is higher than the average; if HRISK is less than one, the risk of the hospital is lower than the average, subject to errors. We can use statistical tests to test the differences between hospitals and identify possible outliers. For example, we can compare the difference in hospital effects between two hospitals and test the hypothesis H0 : ui − u j = 0 (i ≠ j) ; or we can compare the 1 J difference between the i’th hospital and the average of all hospitals by testing the hypothesis H0 : u − u = 0 . i J ∑ j j=1 These statistical tests can be done in GLIMMIX using the ESTIMATE or CONTRAST statement: proc glimmix; class hospital; model y = x / dist=binary link=logit ddfm=bw solution; random hospital / solution; contrast ‘H1 vs H2’ | hospital 1 –1; estimate ‘H1 vs H2’ | hospital 1 -1 / or cl; estimate ‘H1 vs Avg’ | hospital 29 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 –1 —1 –1 / divisor=30 df=29 or cl; run;

In the example, we estimate a random intercept model for hospital profiling. Using the estimation results, we first request to test the difference in hospital effects between the first two hospitals using the CONTRAST statement; then we ask to test the same difference using the ESTIMATE statement. They should provide the same p value, but the ESTIMATE statement provides options for computing the odds ratios (OR) of hospital effects and their confidence limits (CL). The second ESTIMATE statement is used to compare the difference in hospital effects between the first hospital and the average of all hospitals in the sample, assuming the sample consists of 30 hospitals. Since we contrast the effect of one hospital with the average effect, additional options are used, including DIVISOR which divides the coefficients and DF which specifies the denominator degrees of freedom for the test.

INCLUDING HOSPITAL FIXED EFFECTS

The random intercept model discussed above treat the hospital effects as random effects only. It does not contain hospital level (level-2) predictors. Often healthcare researchers are interested in the effects of certain observed hospital attributes on the outcome of care. For example, one may be curious about whether or not the size of a hospital (large, medium, and small) or the type of a hospital (teaching or non-teaching hospital) has an effect on the quality of care. These hospital variables can be readily included in the model. Assume z is a binary variable indicating whether or not the hospital is a teaching hospital. We are interested in finding out if z has significant impact on mortality. For the purpose, we add a level-2 fixed effect to Equations (4),

4

logit(π ij ) = α j + βxij (6) α j = α + γz j + u j

In (6) the hospital intercept α j becomes a linear combination of three terms: a grand mean (α ), a hospital fixed effect ( γ ) and hospital random effects ( u j ). Notice that the differences between the hospitals are now explained by the observed hospital attribute, z, in addition to the random effects. This model in (6) can be written in one equation as

logit(π ij ) = α + γz j + u j + βxij (7)

We can fit the model using the following code: proc glimmix; class hospital z; model y = x z / dist=binary link=logit ddfm=bw solution; random intercept / subject=hospital; run;

The hospital level variable z (teaching status) is a group variable and thus is indicated in the CLASS statement. It is also added to the MODEL statement to specify that it is a fixed effect. The significance of the effect can be evaluated using the result of t-test, which will be provided with the estimated parameter using the SOLUTION option.

INCLUDING CROSS-LEVEL

Researchers are often interested in possible interactions among the explanatory variables. With hierarchical models, we are able to explore the potential interactions among variables across the levels of hierarchy. For example, one might be interested in the effect of interaction between type of the hospital and severity of illness of the patients. Here, we give an example of using a hierarchical model to look at the interaction between teaching hospital and patient risk scores. We specify the hierarchical model as follows,

logit(π ij ) = α j + β j xij

α j = α + γz j + u j (8)

β j = β + θz j

Notice that now the slope is also allowed to vary across hospitals. The third equation specifies that the slope coefficient is a linear combination of the average slope ( β ) and the hospital effect ( z j ). This specification creates a cross-level interaction term in the mixed model,

logit(π ij ) = α + γz j + u j + βxij + θz j xij (9)

where θ is the parameter for the interaction term z j xij . This example shows that cross-level interaction terms can be made by allowing the slope vary across the higher-level units. The hospital-level slope can be a fixed coefficient, as in this case, or it can be specified as a random coefficient by adding random effects to the slope. The models with random intercepts and random slopes are often called random coefficient models (Longford, 1993). The model of Equation (9) is a random intercept model with cross-level interaction. The model can be estimated using GLIMMIX as follows, proc glimmix; class hospital z; model y = x z x*z / dist=binary link=logit ddfm=bw solution; random intercept / subject=hospital; nloptions tech=newrap; run;

5 The SAS code is similar to the previous one, but a new term x*z is add to the MODEL statement. When interaction terms and/or random slopes are included, the model structure becomes more complicated and so is the computation. Thus, the user might need to choose a method of optimization. GLIMMIX makes several optimization methods available and the user can select one through the NLOPTIONS statement. The one shown in this example is NEWRAP (the Newton-Raphson method), which is one of the better optimization methods based on the second derivatives of the variance-. The results of analysis on the CABG data are presented in the next section.

DATA AND RESULTS

The California Office of Statewide Health and Development (OSHPD) develops databases and reports risk-adjusted mortality for Californians undergoing coronary artery bypass grafting (CABG) surgeries. The CABG databases provide data on patients’ pre-operative risk factors (e.g. priority status of procedure, age, left ventricular ejection fraction) and hospital surgical mortality associated with the CABG surgery. The CABG database for 2000 to 2002 contains data on risk factors for more than 50000 patients who had CABG surgery from 83 California hospitals (Park et al., 2005). For the purpose of this study, we took a random sample of 40 hospitals from the database. In this subset, the number of patients is 35526; the number of CABG procedures performed ranges from 119 to 3830 per hospital with an average of 888; the observed mortality of the hospitals ranges from 0.0086 to 0.0756 with a mean of 0.0235.

Table 1. Definition of Variables

VARIABLE DEFINITION

PTSTATUS Dependent variable measured at the patient level nested within the jth hospital PTSTATUS=1 if patient died within 7 days of surgery, = 2 otherwise.

RISK Patient level variable, measuring the preoperative risk of the patient.

TEACH Hospital level dichotomous variable, indicating whether it is a teaching hospital.

For simplicity, we illustrate the application with one patient-level predictor, RISK (risk score or severity index), and one hospital-level covariate, TEACH (teaching hospital status). The response variable, PTSTATUS, is in-hospital mortality, indicating whether or not the patient died within 7 days of surgery. The risk score is a linear function of 39 preoperative risk factors available in the data and estimated using the ordinary logistic regression model. The 39 patient risk factors are defined in the OSHPD public report for the 2000-2002 data available at www.oshpd.ca.gov. Definition of variables used in the hierarchical models is presented in Table 1. The CABG data were used for two applications. In the first application, we fitted a random intercept model with the patient-level predictor and used the model to identify hospital outliers. In the second application, we estimated a hierarchical model with patient-level and hospital-level predictors and a cross-level interaction term. The second analysis was to evaluate the effect of teaching hospital on mortality based on Equation (9).

Results of the first analysis are presented in Table 2 to Table 4. The estimated covariance parameters are shown in Table 2. The estimated variance of the hospital intercepts is 0.0856 with a of 0.0363. The parameter 2 σˆu measures the variability between hospitals. The fact that its estimated value is significantly larger than 0 indicates that there is hospital effect on mortality. In other words, some hospitals performed better than the others even after accounting for patient mix. Table 3 presents the estimated parameters of the fixed effects. The intercept is the average intercept. Coefficient for the risk score has a positive sign and a very low p-value (<0.0001). It indicates that patient’s preoperative risk has a significant effect on mortality.

Table 2. Application 1 – Covariance Parameter Estimates

Parameters Subject Estimate Std Error P-value

2 Hospital 0.0856 0.0363 0.018 INTERCEPT ( σˆu )

6

Table 3. Application 1 - Solution for Fixed Effects

Effect Estimate Standard Error P-value Odds Ratio

INTERCEPT ( αˆ ) 0.0297 0.1059 0.7806 -

RISK ( βˆ ) 0.9964 0.0283 <0.0001 2.71

Table 4. Application 1 – Hospital Profiling Observed Expected Hospital Hospital Sample Mortality Mortality Relative ID Size (%) (%) Risk P-value

01 167 5.39 2.76 1.31 0.27 02 408 3.68 3.40 1.05 0.81 03 301 5.98 2.94 1.56 0.04 04 1225 4.33 3.59 1.20 0.20 05 641 1.87 3.08 0.76 0.16 06 607 1.98 2.99 0.79 0.25 07 3830 2.06 2.07 1.00 0.99 08 463 2.81 3.47 0.88 0.54 09 585 5.13 3.53 1.37 0.08 10 591 1.52 2.43 0.80 0.29 11 985 2.54 2.33 1.07 0.71 12 383 2.87 2.14 1.16 0.51 13 378 3.44 2.12 1.28 0.25 14 672 3.72 3.45 1.06 0.74 15 379 2.11 2.98 0.86 0.48 16 119 7.56 3.77 1.33 0.25 17 902 2.33 2.71 0.90 0.55 18 677 3.40 2.61 1.21 0.30 19 558 3.41 2.93 1.11 0.60 20 582 2.41 2.29 1.03 0.88 21 716 2.51 2.18 1.10 0.63 22 3154 0.86 1.44 0.69 0.02 23 553 2.35 2.40 0.99 0.96 24 1876 1.44 2.12 0.75 0.07 25 1006 1.79 3.59 0.61 0.01 26 1589 3.08 3.17 0.98 0.88 27 733 1.77 2.23 0.88 0.52 28 593 3.37 3.06 1.07 0.71 29 495 4.65 2.27 1.62 0.01 30 418 3.11 2.15 1.21 0.37 31 577 2.08 1.55 1.16 0.49 32 3291 1.49 1.59 0.95 0.69 33 266 3.01 3.12 0.99 0.95 34 908 2.09 2.13 0.99 0.96 35 566 1.41 3.03 0.69 0.08 36 285 2.81 3.24 0.94 0.78 37 379 2.11 2.44 0.94 0.78 38 2098 1.57 1.99 0.83 0.23 39 492 3.46 3.61 0.97 0.89 40 1078 3.06 3.61 0.88 0.40

7

Table 4 presents the results of hospital profiling. The table lists, for each hospital, the sample size, observed mortality, expected mortality, estimated hospital relative risk (HRISK), and p value from the statistical test. As have been discussed, HRISK is a measure of risk due to the hospital relative to the average hospital. Similar to the use of O/E ratio, if HRISK is sufficiently larger than one, the hospital is considered to have performed worse than the average; if it is significantly smaller than one, the hospital is considered to have performed better than the expected. The results suggest that two hospitals (03, 29) performed worse than the average and two hospitals (22, 25) performed better than the average.

Table 5. Application 2 - Solution for Fixed Effects

Effect Estimate Standard Error P-value Odds Ratio

INTERCEPT (αˆ ) 0.0204 0.1174 0.8629 -

0.9782 0.0313 <0.0001 2.660 RISK ( βˆ )

TEACH ( γˆ ) -0.0011 0.2717 0.9968 0.999

RISK*TEACH (θˆ ) 0.0990 0.0727 0.1730 1.104

Table 6. Application 2 – Covariance Parameter Estimate

Parameter Subject Estimate Std Error P-value

2 Hospital 0.0849 0.0355 0.017 INTERCEPT ( σˆu )

Table 5 and Table 6 present the results of the second analysis, which is to evaluate the effect of teaching hospitals and the interaction between teaching hospital and severity of illness. The estimated fixed effects are shown in Table 5. The estimated grand mean ( αˆ ) is 0.0204, which is the average intercept in the model. Estimate of the patient-level fixed effect ( βˆ ) is 0.9782 with a very low p-value (<0.0001), indicating that patient’s risk score is a significant predictor for the outcome. The estimates for teaching hospital ( γˆ ) and the interaction term ( θˆ ) are -0.0011 and 0.0990, respectively. Both have high p values and are not significant at the 0.05 level. It suggests that this particular hospital attribute, teaching hospital status, is not a significant predictor of mortality. It indicates that the relationship between mortality and the severity of illness is not significantly different at teaching and non-teaching hospitals. Table 2 6 presents the estimated variance of the hospital random intercepts ( σˆu =0.0849), which measures the variation in hospital effects. Based on the variance estimate, the odds ratio is 0.557 for moderately low mortality hospital (one below the average) to moderately high hospital (one standard deviation above the average).

CONCLUSIONS

Hierarchical or multilevel models are statistical models for handling data with hierarchical structures. In this paper we have given several examples of hierarchical modeling for health service studies and have shown how to use the new GLIMMIX procedure to fit the models and test hypotheses. SAS GLIMMIX procedure is a highly useful tool for hierarchical modeling with discrete responses. It is versatile, easy to use, and capable of handling large data sets. Although this paper has focused on two-level hierarchical logistic models, it is straightforward to use GLIMMIX to fit data with higher hierarchies and more complex structures.

8 REFERENCES Breslow, N.E. and Clayton, D.G. (1993). “Approximate Inference in Generalized Linear Mixed Models,” Journal of the American Statistical Association, 88, 9-25. Demidenko, E, (2004). Mixed Models: Theory and Applications. Wiley Series in Probability and . Hoboken, New Jersey: John Wiley & Sons, Inc. DeLong, E.., Peterson, E.D., DeLong, D.M., Muhlbaier, L.H., Hackett, S. and D.B. Mark (1997), “Comparing Risk- Adjustment Methods for Provider Profiling”, Statistics in Medicine, 16, 2645-2664. Goldstein, H. (2003). Multilevel Statistical Models (3rd Edition). London: Edward Arnold: New York, Halstead Press. Goldstein, H. and Spiegelhalter, D.J. (1996). “League tables and their limitations: Statistical issues in comparisons of institutional performance,” Journal of the Royal Statistical Society, Series A, 159, 385-409. Hosmer, D.W. and Lemeshow, S. (2000). Applied Logistic Regression (2nd Edition). New York: John Wiley & Sons, Inc. Hox, J. (2002). Multi-Level Analysis. London: Lawrence Erlbaum Associates, Publishers. Little, R.C., Milliken, G.A., Stroup, W.W. and Wolfinger, R.D. (1996). SAS System for Mixed Models. SAS Institute Inc., Cary, NC, USA. Longford, N. T. (1993). Random Coefficient Models. Oxford: Clarendon Press. McCullagh, P. and Nelder, J.A. (1989). Generalized Linear Models (2nd Edition). London: Chapman and Hall. Normand, S.T., Glickman, M.E. and Gatsonis, C.A. (1997). “Statistical Methods for Profiling Providers of Medical Care: Issues and Applications,” Journal of the American Statistical Association, 92, 803-814. Parker, J.P., Z. Li, C.L. Damberg, B. Danielsen, J. Marcin, J. Dai, R. Kravitz, D. Rocke, P. Romano, and A.E. Steimle (2005). “The California Report on Coronary Artery Bypass Graft Surgery 2000-2002 Hospital Data”. San Francisco, CA: California Office of Statewide Health Planning and Development and the Pacific Business Group on Health. SAS Institute (2005). SAS/STAT 9.1 Production GRIMMIX Procedure for Windows. SAS Institute Inc., Cary, NC, USA. Shahian, D.M, MD, Normand, S.T., PhD, Torchiana, D.F., MD, Lewis, S.M., MD, Pastore, J.O., MD, Kuntz, R.E., MD, and Freyer, P.I., PhD (2001). “Cardiac Surgery Report Cards: Comprehensive Review and Statistical Critique,” Ann. Thoracic Surgeons, 72, 2155-68. Snijders, T. A. B., and Bosker, R. J. (1999). Multilevel Analysis. An Introduction to Basic and Advanced Multilevel Modelling. London: Sage.

ACKOWLEDGEMENTS

We are grateful to Drs. Richard Kravitz, Patrick Romano, James Marcin, Beate Danielsen and others for discussions on hierarchical modeling and application. We wish to thank Dr. Alan Zaslavsky for helpful comments on the paper.

CONTACT INFORMATION [email protected] (J. Dai), [email protected] (Z. Li), [email protected] (D. Rocke)

9