University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Leadership Institute Faculty Publications Leadership Institute 2006 Using Multiple Methods in Personality Psychology Brent W. Roberts University of Illinois at Urbana-Champaign, [email protected] Peter D. Harms University of Nebraska - Lincoln, [email protected] Jennifer L. Smith University of Illinois at Urbana-Champaign Dustin Wood University of Illinois at Urbana-Champaign Michelle Webb University of Illinois at Urbana-Champaign Follow this and additional works at: https://digitalcommons.unl.edu/leadershipfacpub Part of the Management Sciences and Quantitative Methods Commons Roberts, Brent W.; Harms, Peter D.; Smith, Jennifer L.; Wood, Dustin; and Webb, Michelle, "Using Multiple Methods in Personality Psychology" (2006). Leadership Institute Faculty Publications. 24. https://digitalcommons.unl.edu/leadershipfacpub/24 This Article is brought to you for free and open access by the Leadership Institute at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Leadership Institute Faculty Publications by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln. Published (as Chapter 22) in Michael Eid and Ed Diener, eds., Handbook of Multimethod Measurement in Psychology (Washington, D.C.: American Psychological Association, 2006), pp. 321–335. Copyright © 2006 American Psychological Association. Used by permission. “This article may not exactly replicate the final version published in the APA edition. It is not the copy of record.” Using Multiple Methods in Personality Psychology Brent W. Roberts, Peter Harms, Jennifer L. Smith, Dustin Wood, and Michelle Webb University of Illinois at Urbana-Champaign In many ways, Campbell and Fiske’s (1959) arti- adequate, and should be avoided at all costs. Or, cle on multitrait-multimethod (MTMM) approaches conversely, we should all be doing multimethod to construct validity has stood like a Platonic ideal studies. This directive fails to consider the empiri- for personality psychologists since its publication. In cal I fact just mentioned, which is that measures of the ideal study, and scientific world, our constructs the same construct seldom correlate highly enough should converge in a robust and coherent fashion across methods to warrant averaging across meth- across diverse methods. Moreover, we should all as- ods (Fiske, 1971). What are needed, rather than pire to use multiple methods in both validating our mandates to perform multimethod studies, are the- measures and in investigating our ideas. Interest- oretical models that successfully incorporate and ingly, that Platonic ideal is not realized as often as explain both the overlap and lack thereof of identi- expected. If one looks closely at the empirical litera- cal constructs across methods. In our following re- ture in personality psychology, the expectation that view, we will attempt to highlight the few theories abstract constructs should converge across methods and empirical examples that have done so. is seldom met at the level implied in the original ar- Our third point is that the focus on multiple ticle. This is not to argue that the Platonic ideal is methods has inadvertently led to a misguided not appropriate. Rather, one of the major points we boondoggle to search for the methodological holy would like to make in this chapter is that the ideal of grail—the one method that deserves our ultimate the MTMM approach is often taken too literally and attention. Campbell and Fiske (1959) should not be is sometimes misused or misinterpreted. Why speak saddled with full responsibility for this phenome- such apostasies? In large part, because we are mo- non beyond the fact that they made it clear that we tivated to reiterate points made, ironically, by Fiske should be pursuing multiple methods. Leave it to himself (Fiske, 1971). human nature that psychologists would take that What are these points? The first is that different idea and try to one up the multimethod approach methods, or modes as Fiske (1971) described them, by finding the ultimate method. Thus, we have are seldom innocuous. Thus, the literal assump- had hyperbolic statements made for and against tion drawn from Campbell and Fiske (1959) that particular methods made since the 1960s. People measures of similar constructs drawn from differ- have argued that self-reports are fundamentally ent methods should converge quite robustly is not flawed and indistinguishable from response styles met as often as we would like. This can lead to er- (Hogan & Nicholson, 1988; Rorer, 1965), that ob- roneous and nihilistic conclusions, such as the con- server ratings are the seat of personality psychol- struct of interest, like depression, does not exist ogy (Hofstee, 1994), that projective tests do not (e.g., Lewis, 1999). The second point is the assump- work (Dawes, 1994), and that we should prioritize tion that monomethod studies are problematic, in- online measures over all other techniques (Kahn- 321 322 ROBERTS ET AL. IN HANDBOOK OF MULTIMETHOD MEASUREMENT IN PSYCHOLOGY (2006) eman, 1999). As will be seen in the following re- Much attention has been dedicated to finding a views, none of these positions is defensible. working taxonomy of traits, and many accept the As the methods used are often tied inextricably Big Five as a minimal number of domains (Gold- to the ideas in a field, we will first provide a work- berg, 1993). We prefer the Big Seven (Benet-Mar- ing definition of the field of personality psychology tinez & Waller, 1997). The Big Seven adds global that will serve as an organizing heuristic for the positive and negative evaluation to the Big Five subsequent review. As will be seen, this is a true and is a better representation of the entire trait do- case of form following function, as the content cate- main. We prefer this model because, as will be seen gories within the field of personality are each dom- later, one distinct characteristic of our definition of inated by specific methods. Then, we review recent personality is the inclusion of reputation as a key multimethod studies within and across the con- element that has been underemphasized in the tent domains of personality psychology. We will field. And although people may not describe them- end with some thoughts about particulars of multi- selves often with terms such as “evil” or “stun- method approaches in personality psychology. ning,” they do describe others in these terms. Motivation, broadly construed, is the second do- main of personality and subsumes all the things WHAT IS PERSONALITY PSYCHOLOGY? that people feel are desirable. We define the domain of motives as what people desire, need, and strive Personality psychology is the study of the indi- for-or perhaps more simply, what people want to vidual differences in traits, motives, abilities, and do. This category includes values, interests, prefer- life stories that make each individual unique (Rob- ences, and goals (e.g., Holland, 1997), in addition to erts & Wood, in press). Figure 1 depicts the pri- the classic notion of motives and needs (e.g., Mur- mary units of focus in our definition of personality, ray, 1938). Currently, this domain is less coherent which reflects what we describe as the neosocioan- than the trait domain because there is no working alytic perspective on personality. For the purposes taxonomy to organize the units of analysis. None- of this chapter, we will focus on the left-hand por- theless, there are striking commonalities across di- tion of the model and forgo a discussion of social verse areas, such as motives, goals, values, and in- roles and culture, so as to focus on the traditional terests. For example, in each of these domains of content and methods of personality psychology. motivation, one can find superordinate themes of As can be seen in Figure 1 there are four “units of agency (desire for status and power) and commu- analysis” or domains that make up the core of per- nion (desire for acceptance and belonging). So, for sonality: traits, motives, abilities, and narratives. example, the primary motivational units have been These four domains are intended to subsume most, achievement, power (agency) and affiliation (com- if not all, of the broad categories of individual dif- munion; Smith, 1992). The higher-order factors that ferences in personality psychology. subsume most value dimensions also reflect power The first domain, traits, subsumes the enduring and affiliation (Schwarz & Bless, 1992). patterns of thoughts, feelings, and behaviors that The third domain reflects abilities and the hier- distinguish people from one another. Or, more eu- archical models identified in achievement litera- phemistically speaking, traits refer to what people tures-that is what people can do (Lubinski, 2000). typically think, feel, or do. In this regard, we view Specifically, intelligence is an individual’s “entire traits from a neo-Allportian perspective (Funder, repertoire of acquired skills, knowledge, learning 1991). From this perspective, traits are real, not fic- sets, and generalization tendencies considered in- tions of people’s semantic memory. They are causal, tellectual in nature that [is] available at any one not just summaries of behavior. Moreover, they are period of time” (Humphreys, 1984, p. 243). Two learned. Even with identical genetically determined models of abilities prevail. The first decomposes temperaments, two individuals may manifest dif- generalized intelligence