urn:nbn:de:gbv:ilm1-2011imeko-029:7 Joint International IMEKO TC1+ TC7+ TC13 Symposium August 31st− September 2nd, 2011, Jena, Germany urn:nbn:de:gbv:ilm1-2011imeko:2 THE QUANTIFICATION OF LATENT VARIABLES IN THE SOCIAL SCIENCES: REQUIREMENTS FOR SCIENTIFIC MEASUREMENT AND SHORTCOMINGS OF CURRENT PROCEDURES Thomas Salzberger Institute for Marketing Management & Institute for Statistics and Mathematics, WU Wien, Vienna, Austria Abstract − In the social sciences, latent constructs play that the applied measurement procedures actually an important role. They appear as explanatory elements in produce valid measures of latent variables. structural theories, or they are of interest as the outcome of Suggesting a quantitative latent variable represents an intervention, for example a support or a preventive pro- a hypothesis built upon a substantive theory of the gramme, a therapy, or a marketing activity. These constructs construct. Specifically, the hypothesis implies an onto- are typically considered to imply a quantitative latent varia- ble that exists independently of measurement. As a matter of logical claim [5]. In science, empirical evidence is routine, measures of latent variables in the social sciences required to corroborate a hypothesis. However, current are treated in the same way as natural scientists handle and practice of measurement in the social sciences typical- utilize their measures. However, in terms of what the con- ly handles the issue of whether or not a latent variable cept of measurement is actually about, the social sciences exists very generously. The predominant paradigm of have veered away from the rigorous concept adhered to in measurement is still based on classical test theory the natural sciences. An arbitrary definition of measurement (CTT) [6], also known as true score theory. CTT nei- and a multitude of procedures which are deemed appropriate ther explains how measurement is accomplished, nor for quantification have resulted in a speculative approach to does it address the ontological claim of a latent varia- measurement. Based on a return to the standard definition of measurement and a new conceptualisation of content and ble. Rather it presupposes the existence of the con- construct validity, the social sciences could advance their struct and deals with associations of scores which are quantitative research substantially. The Rasch model for presumed to be interval-scaled measures. measurement plays an important role in this process. In terms of what the concept of measurement is ac- tually about, the social sciences have veered away Keywords: measurement in the social sciences, con- from the rigorous concept adhered to in the natural struct validity, Rasch model sciences. An arbitrary definition of measurement and a multitude of procedures which are deemed appropriate 1. INTRODUCTION for quantification have resulted in a rather speculative approach to measurement. Based on a return to the In the social sciences, qualitative and quantitative standard definition of measurement and a new concep- research are often seen as competing paradigms. In tualisation of content and construct validity, the social many cases they constitute separate spheres of scien- sciences could advance their quantitative research tific enquiry with limited crossover both in terms of substantially. The Rasch model for measurement plays exchange of ideas and findings and regarding personal an important role in this process. overlap. Even though mixed methods research [1, 2, 3, In the following, a brief overview of current ap- 4] tries to counteract this shortcoming, qualitative proaches to measurement in marketing is provided. considerations often play a limited role in quantitative Subsequently, the meaning of measurement is dis- research, which is generally considered more prestig- cussed and conclusions are drawn. ious, valuable and insightful. The role model of the natural sciences, which have 2. APPROACHES TO MEASUREMENT thrived on quantification, has moulded the purpose of IN MARKETING RESEARCH the social sciences. Medical research, education, busi- ness research etc. are all devoted to measurement and In the social sciences, the vast majority of quantitative modelling. So far, success seems to prove measures of latent variables are inferred from data the social sciences right. Indeed, quantitative insight based on some sort of questionnaires. The data set allows for a more profound understanding of reality. therefore consists of coded responses of persons to However, this is only true provided that the concepts items. Despite being firmly rooted in CTT, measure- investigated really exist as quantitative entities and ment in marketing research has experienced a differ- urn:nbn:de:gbv:ilm1-2011imeko-029:7 Joint International IMEKO TC1+ TC7+ TC13 Symposium August 31st− September 2nd, 2011, Jena, Germany urn:nbn:de:gbv:ilm1-2011imeko:2 entiation leading to a multitude of approaches. CTT is ed scores is almost certainly violated. In practice, still the most popular measurement model. researchers try to avoid floor and ceiling effects by selecting items where the mean respondent score is 2.1. Classical test theory near the scale center and the scores’ distribution is CTT has been made popular in marketing primari- close to normal [11]. However, if this strategy proves ly by Churchill [7], even though empirical work had successful, all items will necessarily capture the same been based on CTT principles before as well. The range of the latent variable resulting in a very narrow fundamental idea of CTT is the separation of the true bandwidth [12]. Furthermore, floor or ceiling effects score and the error score, which are the two compo- may occur as soon as the scale is administered to a nents of the observed score over a number of items. different sample, since the distribution of item scores Today, the congeneric model [8] is the most widely depends on the distribution of the respondents. Thus used variant of CTT. This model applies the logic of reliability at the item and at the scale level is compro- the true score and the error component to the item mised as well. The sample dependence entails that level and explicitly accounts for the latent variable as reliability confounds properties of the instrument the factor score. Borsboom [5] therefore classifies the (error variance) and properties of the sample (true congeneric model as a latent variable theory to be score variance). Hence, a low reliability need not distinguished from the original concept of true score imply a bad scale, specifically when the sample is theory, where the latent variable sits outside the mod- very homogeneous, while a high reliability may be a el. consequence of a heterogeneous sample and/or many The relationship of each item to the latent variable respondents hitting the floor and the ceiling of the is modeled by a linear regression with the latent varia- item score range. The meaning of any given threshold ble being the cause of the manifest score [9]. The for the reliability for a scale to be acceptable, like the association of the manifest scores and the latent varia- often cited 0.7 [10] or any other value, is therefore ble varies in terms of strength, that is some items are limited to a particular sample. It does not transcend more closely related to the latent variable than others. the application of the scale at hand. Since all items are assumed to represent the same The conceptual shortcomings of CTT impact on latent variable, the item scores need to be correlated at the interpretation of alleged respondent measures, as least moderately. The matrix of inter-item correlations well. Since the metric of measures is defined by the is therefore used to estimate the strength of the rela- distribution of person measures, an individual’s meas- tionship between the latent and the manifest variables ure or the mean of a group of respondents can only be by means of factor analysis. Then the factor score, interpreted in comparison to the reference sample. which is inferred from the item scores, is used as the However, it would be much more informative if measure of the latent variable. measures could be referenced back to the measure- The most serious deficiency of CTT as a meas- ment instrument and qualitatively interpreted in terms urement theory lies in the fact that observed item of the items rather than other respondents only. scores are treated as interval-scaled measures. Thus, In summary, CTT presumes what it is supposed to CTT is actually concerned with the behaviour of deliver: measures of latent variables. However, CTT measures of the same concept rather than explaining does not only provide us with doubtful measures. It how we arrive at measures in the first place. Factor also shapes the way measurement instruments are analysis allows the researcher to represent multiple designed in an adverse manner. Specifically, research- replications of measures of the same concept by a ers are not encouraged to elaborate their theory of the single number. On the one hand, this is desirable as latent construct in terms of what more or less of the the factor score is a more parsimonious summary of latent variable implies. Rather CTT favours items the data and it is more precise, as assessed by reliabil- which are perfect replications of one another. ity [10]. On the other hand, the question whether indi- vidual items really represent measures and whether the 2.2. Formative models latent variable actually exists is not even remotely In CTT, causality is assumed to flow from the la- addressed. This disqualifies CTT as a scientific theory tent variable to the indicators, which are therefore of measurement. referred to as reflective indicators [9]. In theory, there The application of CTT to item scores that are ob- are indefinitely many potential indicators and it does viously not measures but the result of an interaction of not matter which items are used. It has been argued respondents and items is problematic and essentially that in marketing many constructs are different inas- unjustified.