Implicit and Explicit Attitudes 1 A multitrait-multimethod validation of the Implicit Association Test: Implicit and explicit attitudes are related but distinct constructs Brian A. Nosek University of Virginia Frederick L. Smyth University of Virginia Corresponding Author: Brian Nosek Department of Psychology Box 400400; 102 Gilmer Hall University of Virginia Charlottesville, VA 22904-4400 Em: [email protected] Word count = 8,014 Implicit and Explicit Attitudes 2 Abstract Recent theoretical and methodological innovations suggest a distinction between automatic and controlled evaluative processes. We report a construct validation investigation of the Implicit Association Test (IAT) as a measure of attitudes. In Study 1, a composite of 57 unique studies (total N=13,165), correlated two-factor (implicit and explicit attitudes) structural models fit the data better than single-factor (attitude) models for each of 57 different domains (e.g., cats-dogs). In Study 2, we distinguished attitude and method factors with a multitrait-multimethod design: N=287 participants were measured on both self-report and IAT for up to seven attitude domains. With systematic method variance accounted for, a correlated two-factor-per-attitude- contrast model was again superior to a single-factor-per-attitude specification. We conclude that these implicit and explicit measures assess related but distinct attitude constructs. Abstract = 125 words Keywords = Implicit Social Cognition, Attitudes, Individual Differences, Construct Validity, Structural Equation Modeling Implicit and Explicit Attitudes 3 Realizing that the human mind is more than the sum of its conscious processes, a number of theorists have proposed a conceptual distinction between evaluations that are the products of introspection, called explicit attitudes, and those that may exist outside of conscious awareness or conscious control, called implicit attitudes (Greenwald & Banaji, 1995; Wilson, Lindsey, & Schooler, 2000). Greenwald and Banaji (1995, p. 8) defined implicit attitudes as “introspectively unidentified (or inaccurately identified) traces of past experience that mediate favorable or unfavorable feelings toward an attitude object.” This theory has developed in conjunction with the invention of measurement tools that assess automatic evaluative associations without requiring introspection (e.g., Fazio, Sanbonmatsu, Dunton & Williams, 1986; Greenwald, McGhee, & Schwartz, 1998; Nosek & Banaji, 2001; Wittenbrink, Judd, & Park, 1997). Some experiences with these new measurement tools have spawned doubts about whether they measure attitudes at all (Karpinski & Hilton, 2001), and whether a conceptual distinction between implicit and explicit attitudes is worthwhile (Fazio & Olson, 2003). The purpose of the research documented in this paper is to critically examine whether a conceptual distinction between implicit and explicit attitudes is warranted. The conceptual and empirical justification for psychological constructs requires development of a “nomological net” of facts, relationships, and validity evidence that clarifies the identity and utility of the construct (Cronbach & Meehl, 1955; McArdle & Prescott, 1992). The nomological net supporting the validity of implicit attitudes has been gaining strength (Greenwald & Banaji, 1995; Nosek, Greenwald, & Banaji, in press; Wilson, Lindsey, & Schooler, 2000). For example, Poehlman, Uhlmann, Greenwald, and Banaji (2004) conducted a meta-analysis of 61 studies examining the predictive validity of the Implicit Association Test (IAT; Greenwald et al., 1998), a measure thought to be influenced by automatic associations, and found that the IAT had robust predictive validity across domains, and outperformed self- Implicit and Explicit Attitudes 4 report measures in some domains (stereotyping and prejudice), while self-report outperformed the IAT in other domains (e.g., political preferences). Also, recent social neuroscience research finds evidence for a neurological distinction between automatic and controlled evaluative processes (Cunningham et al., 2003; Cunningham et al., 2004). The present research provides another avenue of evidence for this budding nomological net by examining the relationship between implicit and explicit attitude measures to determine whether they can be fairly interpreted as measuring a single construct, or whether they assess related, but distinct constructs. Greenwald and Farnham (2000) observed that a model describing implicit and explicit self-esteem as distinct factors provided a better fit than a single self-esteem conceptualization. Likewise, Cunningham, Preacher, and Banaji (2001) found implicit and explicit measures of racial attitudes to reveal related, but distinct factors. In Study 1, we test whether this observation can be generalized across attitudinal domains. Consistent with our hypothesis, across domains, a conceptualization of implicit and explicit measures as assessing related but distinct factors provided a better fit to the data than a single attitude conceptualization. In Study 2, guided by principles articulated by Campbell and Fiske (1959), we use a multitrait-multimethod design and comparative structural modeling analyses to transcend some of the inferential limitations of previous research. This approach allowed us to distinguish attitude and method variance from IAT and thermometer ratings, the operationalizations of implicit and explicit attitudes respectively. In this approach, we demonstrate (a) convergent and discriminant validity of the IAT, (b) that a model of distinct, but related implicit and explicit attitudes best fits the data, and (c) that this characterization is not attributable to attitude- irrelevant method variance of the IAT or of self-report. Study 1 When accounting for the structure of data collected by IAT and self-report, the superiority of a correlated two-factor model (implicit and explicit attitudes) to a single attitude Implicit and Explicit Attitudes 5 model has been demonstrated for a few attitudinal domains, including racial attitudes, ethnocentrism, and self-esteem (Cunningham et al. 2001; Cunningham, Nezlek, & Banaji, 2004; Greenwald & Farnham, 2000). Here we test the generalizability of this finding for each of 57 web-based studies, each of which included an IAT and self-reports concerning a different attitude object pair. Like the prior research, specification of implicit and explicit attitude constructs in Study 1 is confounded with measurement method. As a consequence, a two-factor solution is an indeterminant function of both attitude and method variance.1 So, while this study establishes the generality of previous research, it does not rule out a method variance explanation for the effects. Study 2 addresses this limitation with a multitrait-multimethod design. Method The materials and procedures for Study 1 were described in detail by Nosek (2005). 2 The study was administered online via Project Implicit (2002) between January 10 and October 20, 2003. Participants 7,853 volunteers completed a total of 15,248 sets of implicit and explicit measures for one of 57 attitude object pairs. Following data cleaning (see Nosek, 2005), a total of 13,165 usable completed tasks remained (average n of 231 per attitude pair). Materials Implicit Association Test (IAT). IATs for 57 attitude object pairs were selected to represent a wide range of topics (see Table 1). The IAT measures association strengths between category (e.g., Democrats-Republicans) and attribute (e.g., Good-Bad) concepts. IAT effects are expressed with the D measure described by Greenwald et al. (2003). For all tasks, positive D scores indicate implicit preference for the attitude object that was preferred on average. Two D scores (one based on ‘practice’ trials, and the other based on ‘critical’ trials) served as the Implicit and Explicit Attitudes 6 indicators of implicit attitudes (see Greenwald et al., 2003). Across the 57 domains, median internal consistency for IAT split-halves was r=.63. Self-report measures. Two self-reported attitude items from the questionnaire reported by Nosek (2005) were of direct relevance for this study. Feelings of warmth were indicated on a 9-point scale for each attitude object of a pair (e.g., toward Democrats and toward Republicans). Participants also used a 9-point scale to rate their preference for the attitude objects compared to the perceived norm.3 A difference score was calculated between ratings of each object of a pair so as to be conceptually parallel to the relative assessment property of the IAT (e.g., the warmth rating of Republicans subtracted from that for Democrats). Positive values indicate greater liking for the object that was implicitly preferred on average (-8 to +8). Median internal consistency between items was r=.74. Procedure Participants first registered an identity and completed a demographics questionnaire (http://implicit.harvard.edu/implicit/research/). Once registered, participants could log-in at the front page of the research web site and be randomly assigned to one of many possible studies in the Project Implicit study pool. Participants who logged in multiple times were never assigned to the same study more than once. The 57 attitude object pairs described here were treated as 57 independent studies. Implicit and explicit measures for each domain were presented in a randomized order. Additional description is available in Nosek (2005). Analyses Our primary hypothesis was that the two measurement approaches, self-reports and