HGLM Rasch Model 1 One-Parameter Hierarchical Generalized Linear Logistic Model: An Application of HGLM to IRT Akihito Kamata College of Education Michigan State University* This paper was presented at the annual meeting of American Educational Research Association, San Diego, CA, April, 1998. * The author is now at: College of Education Florida State University 307 Stone Building Tallahassee, FL 32306-4453 (850) 644-8794 [email protected] HGLM Rasch Model 2 Abstract In this paper the Rasch model is generalized as a special case of the hierarchical generalized linear model (HGLM). Parameter recovery study reveals that (1) item parameters are estimated properly, and (2) the variance of the person parameters are estimated properly. Extensions of the generalized model are also shown. Extensions of the generalized model include (1) one-step analysis of test data with person-level predictors, (2) a DIF model , (3) three-level model, and (4) multidimensional Rasch models. HGLM Rasch Model 3 A two-step analysis using item response theory (IRT) models is common practice, especially in investigating effects of student characteristics on student abilities. In such a two- step analysis, student abilities are estimated via a standard IRT model as the first step. Then, in the second step, ability estimates are used as an outcome variable, and student characteristic variables are used as predictors in a simple linear model, such as multiple regression and analysis of covariance. However, such a two-step analysis may not provide accurate results, because of at least two reasons. First, the standard errors of ability estimates from an IRT model are heteroscedastic. When ability estimates are used as an outcome variable, this results in non- random errors of measurement of the dependent variable. Second, it is known that person parameter estimates from marginal maximum likelihood estimation are biased and inconsistent (Goldstein, 1980). It would be more reasonable to perform a single analysis by including student characteristic variables as predictors, i.e., linear constraints, in an IRT model (Zwinderman, 1991). Through such a single analysis rather than a two-step analysis, one can expect improved estimation of the effects of such linear constraints themselves on a latent trait, because effects of linear constraints are estimated simultaneously with ability parameters. As a result, the heteroscedastic nature of the standard errors of ability estimates, as well as unbiasedness and inconsistency of the estimates, is taken into account. Similarly, one can expect improved precision for estimates of item- and person-parameters (Mislevy, 1987). Attempts to generalize IRT models by adding linear constraints were made by several authors. Fischer (1983), for example, generalized the standard binary Rasch model by decomposing an item difficulty parameter into linear combinations of an item parameter and one or more person-varying parameters. This approach enables us to include person characteristic HGLM Rasch Model 4 variables as linear constraints in the Rasch model. Furthermore, this approach also has been applied in measuring change in unidimensional traits. In such application any change of person parameter occurring between time points is described as a change of item parameters instead of change of the person parameter. Similarly, Bock, Muraki and Pfeiffenberger (1988) extended the 3-parameter logistic model by adding a variable of time points that a specific item is tested. It is a “growth model” of item difficulties. Their approach was specifically used for detecting item- parameter drift across multiple time points. Linacre (1989), on the other hand, added an indicator variable for “raters” as a linear constraint to polytomous Rasch models in order to detect different degrees of severity between raters. Linacre viewed items, examinees and raters as different facets and called the model a many-facet Rasch model (Linacre, 1989). However, one critical limitation of the above generalizations is that they deal with all parameters, including item- and person-parameters and linear constraints, as fixed parameters. As a result, models always have to be formulated within a single level. More recently, Adams and Wilson (1996) proposed a model with linear constraints, i.e., a random coefficient multinomial logit model (RCMLM), that is general enough to include a wide range of Rasch models, both dichotomous and polytomous models. Adams, Wilson and Wang (1997a) further generalized the RCMLM to its multidimensional form (MRCMLM). The RCMLM and MRCMLM are formulated so that person parameters are random variables. Adams, Wilson and Wu (1997b) explicitly recognized the RCMLM as a multi-level model, in which person-characteristic variables can be added as fixed parameters that are related to a latent trait. This was the first time that a regular IRT model was conceptualized as a multi-level model. However, their approach was limited to a two-level formulation, i.e., the model is only able to HGLM Rasch Model 5 include person-varying variables as linear constraints. This study will show another way to model the Rasch model as a multi-level model. I take an approach to generalize the Rasch model as a special case of the hierarchical generalized linear model (HGLM) (Raudenbush, 1995; Stiratelli, Laird, & Ware, 1984; Wong & Mason, 1985). The HGLM is an extension of the generalized linear model (GLM) (McCullagh & Nelder, 1989) to hierarchical data. This study, accordingly, treats item response data as hierarchical data, where items are nested within people. It should be strongly noted that estimating person- and item- parameters per se through HGLM is not the purpose of this generalization. Yet, it is essential for the model to be able to estimate parameters appropriately with the simplest model, i.e., the Rasch model itself, in order to be further extended to more complex models. Therefore, this study puts a great deal of effort to show the equivalence between the generalized model and the Rasch model, both algebraically and numerically. Then, I will briefly present how the reformulated model can be extended to a model with a person-level predictor variables, and a model with more than two levels, and a multidimensional Rasch model, . Model First, the standard binary-response Rasch model is presented for the purpose of making a clear connection with the HGLM. Then, the Rasch model is carefully reformulated using the HGLM framework. Specifications of the HGLM framework include a sampling distribution of item responses, its expectation and variance, a link function, a level-1 structural model, and level-2 HGLM Rasch Model 6 models. The Standard Rasch Model Let pij be the probability that the person j (j = 1, … , n) gets the item i correct, qj be the latent trait of the person j, di be the difficulty of the item i (i = 1, … , k), and yij be a binary outcome, indicating a score for the jth person on the ith item (yij = 1 if the person answers the item correctly, and yij = 0 if the person answers incorrectly). Then, the conditional distribution of the outcome yij, given pij, is a binomial distribution with parameters 1 and pij, which is also known to be a Bernoulli distribution with a parameter pij. Specifically, yij | pij ~ B(1, pij ) . (1) Based on the above probability model, the Rasch model is defined to be exp[q j - d i ] 1 pij = = , (2) 1+ exp[q j - d i ] 1+ exp[- (q j - d i )] which is equivalent to stating æ p ö ç ij ÷ logç ÷ = q j - di . (3) è1- pij ø In the above Rasch model, parameters qj and di are considered to be fixed, and there are n + k - 2 HGLM Rasch Model 7 parameters to be estimated. However, the number of parameters to be estimated will be reduced to 2k + 1 because, in the Rasch model, the number of items answered correctly is a sufficient statistics, and people who get the same raw score will have the same ability level. As a result, there are k + 1 possible unique scores on a k-item test plus k item parameters. As already mentioned, attempts to reformulate the Rasch model in terms of a linear model have been made by several other authors. For example, Fisher’s parameter estimation method (1983) are based on conditional maximum likelihood estimation (CMLE) (Andersen, 1972), in which both qj and di are considered to be fixed parameters. The CMLE is based on sufficient statistics for the Rasch model, i.e., the number of correct responses for each person and each item. One advantage of the CMLE is that the likelihood function does not contain q = (q1, ... , qj), yet it produces consistent and efficient estimation of item parameters. Also, the CMLE does not require any assumptions about the population distribution of q values, i.e., both item- and person- parameters are considered to be fixed parameters. However, this approach is strictly limited to a family of the Rasch model. An extension of the standard Rasch model is a model that considers the latent trait qj to be a random variable. In other words, it is assumed that the examinees represent a random sample from a population in which ability is distributed according to a specified density function. In general, the density function of a latent trait is a function of q, given t, where t is the vector containing the parameters of the examinee population ability distribution g(q | t) . (4) HGLM Rasch Model 8 Often, the standard normal distribution, q ~ N (0, 1) , is used for the density function g, but it could have other forms, of course. Item- and person-parameters based on this assumption are estimated via marginal maximum likelihood estimation (MMLE) (Bock & Aitkin, 1981; Bock & Lieberman, 1970). To date, the MMLE is considered to be the only one of the likelihood methods that makes use of the population distribution of q.