The Rasch model and time-limit tests an application and some theoretical contributions Arnold L. van den Wollenberg THE RASCH MODEL AND TIME-LIMIT TESTS an application and some theoretical contributions Promotor: Prof. Dr. E.E.Ch.I. Roskam THE RASCH MODEL AND TIME-LIMIT TESTS an application and some theoretical contributions PROEFSCHRIFT TER VERKRIJGING VAN DE GRAAD VAN DOCTOR IN DE SOCIALE WETENSCHAPPEN AAN DE KATHOLIEKE UNIVERSITEIT TE NIJMEGEN OP GEZAG VAN DE RECTOR MAGNIFICUS PROF. DR. Ρ G.A.B. WIJDEVELD VOLGENS BESLUIT VAN HET COLLEGE VAN DECANEN IN HET OPENBAAR TE VERDEDIGEN OP DONDERDAG 22 NOVEMBER 1979, DES NAMIDDAGS TE 4.00 UUR door ARNOLDUS LAMBERTUS VAN DEN WOLLENBERG geboren te Nijmegen 1979 DRUK. STICHTING STUDhNTENPERS NIJMEGEN Aan Els, Linda en Janneke CONTENTS 1. The Rasch mode I 1 1.1. Introduction 1 1.2. The general latent structure model 8 1.3. The logistic model of Rasch 11 1.4. Parameter estimation 18 1.5. Testing procedures for the Rasch model 28 1.5.1. The conditional likelihood ratio test 28 1.5.2. The Fischer and Scheiblechner statistic S 31 1.5.3. The Martin Löf statistic Τ 34 1.5.4. The Wright - Panchapakesan statistic Y 37 1.5.5. Model fit and statistical testing 39 1.6. Extensions of the model 40 1.7. Conclusion 43 2. Application of the Rasch model to the ISI time-limit intelligence tests 45 2.1. Introduction 45 2.2. The ISI tests 49 2.3. Model test high-low 53 2.3.1. ISI-1 (high-low) 55 2.3.2. ISI-2 (high-low) 57 2.3.3. ISI-3 (high-low) 57 2.3.4. ISI-4 (high-low) 58 2.3.5. ISI-5 (high-low) 58 2.3.6. IS 1-6 (high-low] 58 2.4. Model tests based on speed partitioning 59 2.4.1. ISI-1 (speed partitioning) 61 2.4.2. ISI-2 (speed partitioning) 64 2.4.3. IS1-3 (speed partitioning) 64 2.4.4. ISI-4 (speed partitioning) 65 2.4.5. ISI-5 (speed partitioning) 67 2.4.6. ISI-6 (speed partitioning) 67 2.5. The ISI tests and Rasch homogeneity conclusions 68 2.6. The guessing and skipping assumptions 70 2.7. Speed and precision 72 2.7.1. The constancy hypothesis 74 2.7.2. The independence of speed and precision 76 2.7.3. The dimensionality of precision 77 Testing the Hasch model 83 3.1. Introduction 83 3.2. Data construction 84 3.3. Violation of the axioms 88 3.3.1. Violation of monotonicity 89 3.3.2. Violation of sufficiency of simple sun statistics 90 3.3.3. Violation of the unidimensionality axiom 94 3.3.4. Conditions for failure of the likelihood ratio test in detecting multidimensionality 95 3.3.5. When multidimensionality becomes manifest in the likelihood ratio test 101 3.4. Alternative testing procedures 107 3.4.1. Partitioning criteria 108 3.4.2. A possible artefact 112 3.4.3. Alternative test statistics 114 3.4.3.1. Some prerequisites 114 3.4.3.2. The Q1 model test 118 3.4.3.3. The Q2 model test 126 3.5. Conclusion 130 The ISI tests revisited 133 4.1. Introduction 133 4.2. Problems in the application of Q1 and Q, 134 4.3. The correlation between parameter estimates as a descriptive measure of goodness of fit 138 4.4. Sample size and the information measure 142 4.5. Results for ISI-1 145 4.5.1. ISI-1; the restricted sample S^ 145 4.5.2. ISI-1; the restricted samples S., to S™ 148 4.5.3. ІЫ-1; the Q- model test 155 4.5.4. ISI-1; conclusions 156 4.6. Results for ISI-2 156 4.6.1. ISI-2; the restricted sample S... 156 4.6.2. ISI-2; the restricted samples S^ to S?n 157 4.6.3. ISI-2; the Q, model test 161 4.6.4. ISI-2; conclusions 163 4.7. Results for ISI-3 153 4.7.1. ISI-3; the restricted samples S·., to S7n 103 4.7.2. ISI-3; the Q2 model test 166 4.7.3. ISI-3; conclusions l^y 4.8. Results for ISI-4 167 4.8.1. ISI-4; the restricted samples S... to S7n 167 4.8.2. ISI-4; the Q2 ncdel test 168 4.8.3. ISI-4; conclusions 169 4.9. Results for ISI-5 171 4.9.1. ISI-5; the restricted samples S^ to S20 171 4.9.2. ISI-5; the Q2 model test 172 4.9.3. ISI-5; conclusions 173 4.10. Results for ISI-6 173 4.10.1. ISI-6; the restricted samples S.. to S,« 173 4.10.2. ISI-6; the Q2 model test 176 4.10.3. ISI-6; conclusions I77 4.11. The ISI tests and Rasch homogeneity; conclusions 178 4.12. The assessment of subject paramete'rs 180 4.13. The dimensionality of precision 185 Summary 191 Samenvatting 195 Appendix A 199 Appendix Б 209 Glossary 213 References 217 THE RASOI MODEL 1 1.1 Introduation In the field of psychometrics two important approaches can be distinguished. Of these two classical test theory is the most widely known. The second approach, latent structure analysis, is of more recent date. Well known handbooks in the tradition of classical test theory are those of Gulliksen (1950), Guilford (1954) and Nunnally (1959). The book of Lord and iNovick (1968) also contains an exten­ sive treatment of the classical model. The basic framework of classical test theory is the general linear model (e.g. Bock and Wood (1971), Lunsden (1976)). Various kinds of regression techniques are used; multivariate analysis techniques and especially factor analysis are closely related to the classical model. In contrast to classical, statistical test theory, latent struc­ ture analysis aims at (in the terminology of Coombs, 1964) 'inferen­ tial classification of stimuli and persons', based on theoretical assumptions concerning the structure of test data and on theories of scaling and measuranent. The model introduced by Rasch (1960), known as the (dichotomous) Rasch model or the simple logistic model, will be the central theme of this study. The Rasch model belongs to the class of latent trait models, which class in turn belongs to the wider class of latent structure models. To get a good appreciation of the place of the Rasch model, it seems useful to pay some attention to latent struc­ ture theory in general. Without suggesting the existence of schools, one can say that in latent structure theory three mainstreams can be distinguished. To the first mainstream the name of Lazarsfeld is strongly associated. He was the first to develop latent structure analysis, the first publications on the subject matter date back to 1950 (Lazarsfeld, 1950a, 1950b). In latent structure analysis a distinction is made between (hypo­ thesized) latent variables and (observed) manifest variables. The latent class and latent profile models assme nominal latent variables and are therefore called qualitative models. Other models assume latent variables of at least ordinal level; these models are called quanti­ tative. In the research tradition initiated by Lazarsfeld the quali­ tative models get the bulk of the attention. Ihe book of Lazarsfeld and Henry (1968) gives an inventory of latent structure analysis up to that time. Again the qualitative models get the bulk of the attention, but also quantitative models such as latent distance analysis, the polynomial traceline model and the latent content model are discussed and even the models of Birnbaum (1958) and Rasch (1960) are referred to. The names of Bimbam and Lord are associated to the second line of research in latent structure theory. Birnbaum (1958) introduced a latent trait model based on the logistic probability function: exp iDa (ξ-σ ), r.CO = 1 . (1.1) 1 + exp{Da (ξ-σ )} In this function σ is the item difficulty and α is the discrimina­ ting power; ς is the subject ability parameter. The constant D is a scaling factor that can be chosen freely. In his contribution to Lord and Novick (1968), Birnbaum (1968) also discusses the so called three parameter logistic model: expiDa (ζ-σ)} f, (О = У + (1-rJ i , (1.2) 1 1 1 1 + expíDa (ζ-σ )} where γ is a guessing parameter. The name three parameter logistic model stems from the fact that every item is characterized by three parameters. The probability function (1.1) is also called the two parameter logistic model. When the probability of a correct response 2 by mere guessing equals zero, the three parameter model (1.2) reduces to the two parameter model (1.1). Λ generalization of the Birnbaum model to more than two response categories was presented by Bock (1972). Lord (1952, 1953a, 1953b) drew attention to a latent trait model based on the normal ogive, that was originally introduced by Lawley (1943). Whereas one could say that Lazarsfeld had been inspired by the problems of attitude measurement, the second line of research is clear­ ly initiated from the field of ability measuranent. The third line of research and theorizing had been initiated by Rasch (1960, 1961, 1966a, 1966b, 1966c, 1966d). Independently of latent structure analysis, he developed a measurement theory for ability, which led to a model that is mathematically a special case of Bim- baun's two parameter logistic model. However, the essential properties of the Rasch model, population independence and specific objectivity (see below), do not hold for the model of Birnbaum.