To appear in Handbook of Individual Differences Tomas Chamorro-Premuzic, Adrian Furnham and Sophie von Stumm (Eds.) This version is the final, as submitted version and will differ from the published version. Methodological Advances in Differential Psychology William Revelle David M. Condon Joshua Wilt Northwestern University Methods for differential psychologists are the methods of all scientists: describe and test mod- els of data. Our field is distinguished by the nature of our data and the specialized tools we use for analysis. For the differential psychologist, data come from self-report, from observations, from physiology, and from behavioral residues. Data are recorded over time and over space. The challenges of collecting data are limited only by our imagination. Methods of analysis emphasize model fitting and model evaluation. The goals of methods in Differential Psychology are no different from those of any other science: de- Data = Model + Error (1) scriptive and testable explanations of phenomena. A revolution in data analysis has occurred over Methods thus involve the collection and analysis of the past thirty years: the recognition that we model data. What distinguishes scientific fields from each data and compare alternative models to each other other, and the field of differential psychology in par- (Rodgers, 2010). This approach is summarized in ticular is what constitutes data, the theories of our Equation1 which, if we recognize that our error is data, and the analytical techniques used to describe someone else’s signal, is better expressed as Equa- and model data. This chapter is divided into two tion2: main sections: the kinds and sources of data we col- Data = Model + Residual: (2) lect and the ways in which we model (analyze) the data. In that entire text books are devoted to data The process of research then is one of finding mod- collection, to design (Shadish et al., 2001), to in- els that fit the data with acceptably small residual ference (Pearl, 2000), and to each of many ways to values. “Models, of course, are never true, but for- model data (Judd et al., 2009; Loehlin, 2004; McAr- tunately it is only necessary that they be useful. For dle, 2009; McDonald, 1999; Mulaik, 2010; Rasch, this it is usually needful that they not be grossly 1960), this review will necessarily be of the basic wrong.” (Box, 1979, p 2). The current approach concepts rather than the specifics of particular meth- goes beyond just asking for usefulness by asking if ods. For a thorough discussion of research meth- the specified model is better than alternative models ods of individual differences that is limited to per- (Rodgers, 2010). sonality narrowly defined (e.g, not including intelli- gence, interests or values) see the handbook edited Coomb’s Theory of Data by Robins et al.(2007). and Cattell’s Data Box The left hand sides of Equations1 and2 are Data. What are the data that we collect? At an abstract level, data can be organized along three different di- contact: William Revelle [email protected] mensions: type of comparison (order versus prox- Draft version of December 20, 2010 imity), the elements being compared (people, ob- This is the precopy edited draft. The final version will jects, people x objects) and the number of compar- differ slightly. isons (one or more) (Coombs, 1964). Within this framework, a person can be said to be more than an object (e.g., if passing an ability test item) or 1 2 REVELLE, CONDON, WILT to be near an object (if endorsing an attitude item), behave, these measures of trait Neuroticism, Ex- and one person can prefer one object to another ob- traversion, Agreeableness, Conscientiousness, and ject (be closer to one attitude than another) or have Openness show stability over long periods of time a stronger preference than someone else. People and correlate with suitable behavioral observations can also differ in the way they group objects. The and other reports (Roberts et al., 2007). In contrast Coombs(1964) model continues to be used within to measures of ability, these items are thought to psychometrics by virtue of the distinction between measure typical performance. In other words, they ability and preference items in item response the- measure how one usually thinks, feels and behaves ory (Chernyshenko et al., 2007) and in terms of in- rather than how well one can think. dividual differences in multidimensional scaling of A similar example would include self-report situational stress. items that allow inference about the internal states Cattell’s data box (Cattell, 1946) emphasized of Energetic Arousal or Tense Arousal (Schimmack three sources of data: People, Tests, and Occasions & Reisenzein, 2002; Thayer, 2000). When asked and considered how correlations can be taken be- about energetic arousal (how alert, active or vigor- tween tests and across people at one occasion (R ous one feels in contrast to sleepy, tired or drowsy) analysis), just as correlations can be found between or tense arousal (anxious, worried or tense versus people across tests (Q analysis), or tests can be cor- calm or relaxed), subjects’ scores will change over related within people across occasions (P analysis), the day and in response to factors such as caffeine, etc. Subsequently, Cattell(1966) expanded the data exciting or depressing movies, and exercise (Rev- box to include Background or preceding variables elle, 1993). as well as Observers. The data box concept has been These items are direct and obvious. They may used throughout differential psychology to demon- be formed into scales using factorially homogenous strate the many ways of analyzing data, but the pri- keying (Goldberg, 1972), also known as an induc- mary influence has probably been on those who tive strategy (Burisch, 1984). Classic examples of study personality and cognitive development and such inventories are the Eysenck Personality Inven- change over the life span (McArdle & Bell, 2000; tory (EPI, Eysenck & Eysenck, 1968), the NEO-PIR Mroczek, 2007; Nesselroade, 1984). (Costa & McCrae, 1985), and the sixteen Personal- ity Factors (16PF, Cattell & Stice, 1957). Some in- Methods of data ventories, however are developed using the empiri- collection cal or external strategy of finding items that distin- guish known groups from people in general, e.g., Individual differences can be assessed by asking the MMPI (Hathaway & McKinley, 1943) or the people about themselves (their identity) and other Strong Vocational Interest Inventory (Strong, 1927). people (their reputation) or by observing behav- They also differ from rational or deductively con- ior (what people or other animals do), physiology structed tests such as the California Psychological and behavioral residues. Of these, the predominant Inventory (CPI, Gough, 1957) or the Personality method is probably that of self report, through the Research Form (PRF, Jackson, 1967). use of either questionnaires, projective instruments, or narratives. The advantages and disadvantages of empiri- cal, rational, and homogenous keying techniques Self report were well reviewed by Goldberg(1972), Hase & Goldberg(1967) and Burisch(1984). In general, “Do you get angry easily?”, “Do you find it diffi- rational and factorial techniques work better for cult to approach others?”, “Do you make people feel predicting more predictable criteria, but empiri- at ease?”, “Do you do things according to a plan?”, cal/external techniques are better able to predict “Do you carry the conversation to a higher level?”. very unpredictable criteria (e.g., dropping out of These are typical self report items taken from the college). Tests assessing interests (Holland, 1959, International Personality Item Pool (IPIP, Goldberg, 1996; Strong, 1927) have traditionally used empiri- 1999). They follow the basic principle that if you cal scale construction methods and have incremen- want to know something about someone, ask them. tal validity when predicting diverse criteria such as With the instruction to answer the way you normally success in graduate school (Kelly & Fiske, 1950). METHODS IN DIFFERENTIAL PSYCHOLOGY 3 Some question how self reports can be valid one’s life and, like any good story, convey a variety given the tendency to dissimulate or self enhance. of themes through its structure, characters, and plot R. Hogan & Nicholson(1988), R. Hogan & Kaiser (McAdams, 1993). (2005), and R. Hogan(2007) address this issue for predicting real life criteria (leadership effective- Due to the massive amount of scenes, events, ness in organizations). Self report measures are and memories a person accumulates throughout a quite successful at predicting this important crite- lifetime, quantitative analysis of narrative identity rion. J. Hogan et al.(2007) directly address the at first seems a daunting undertaking. Indeed, the problem of faking and report that it was not a prob- cumbersome methods of the case study and the lem for selecting job applicants for security posi- study of single lives are more amenable to qual- tions. itative analysis. However, modern narrative re- searchers have been up to the task, as the past two Constructing self report inventories. Practical decades have seen steady growth in creative, quan- advice for constructing self report inventories for titative methodologies to analyze narratives. the differential psychologist (e.g., Clark & Wat- son, 1995; Simms & Watson, 2007; Watson, 2005) One fruitful approach to dealing with the prob- emphasizes starting with a good theoretical under- lem of scene selection is the introduction of the standing of the constructs to be measured and the standardized life story interview (McAdams et al., population of interest, writing items that are clear 1997) in which people narrate a set of important and readable, examining the internal structure of the scenes in their lives (high points, low points, turn- items, purifying the scales developed, checking for ing points, vivid memories from childhood, ado- external validity in terms of correlations with crite- lescence, adulthood, and an imagined future scene) rion groups, further refinement of items and finally and trained human coders assess these scenes for extensive documentation.