What Is Beyond the Big Five? Plenty!

Sampo V. Paunonen Douglas N. Jackson University of Western Ontario

ABSTRACT In a recent analysis of personality data, Saucier and Goldberg (1998) sought to answer the question, What is beyond the Big Five? Those authors evaluated numerous clusters of English person-descriptive adjectives that have been suspected of referring to non–Big Five dimensions of personality. Their results led them to conclude that most, if not all, traits of personality can be adequately subsumed within the Big Five factor space. In contrast, our reanalysis of Saucier and Goldberg’s own data, using a more realistic criterion for deciding on whether a variable does or does not fall within a particular factor space, contradicts their claim. We are led to the conclusion that there are plenty of dimensions of behavior beyond the Big Five.

Few topics in contemporary psychology have generated as much research and theoretical interest as has the Five-Factor Model of personality. That model has been embraced not only by a personality psychologist, but by researchers in clinical, industrial/organizational, and developmental psychology as well. The Five-Factor Model, of course, posits that there is a structure to individual differences in human behavior, such that the traits of personality can be reduced to five orthogonal factors of personality—the so-called Big Five.

This research was supported by the Social Sciences and Humanities Research Council of Canada Research Grant 410-98-1555 to Sampo V.Paunonen. WethankMichael Ashton and Kibeom Lee for their comments on this article. Correspondence concerning this article should be addressed to Sampo V. Paunonen, Department of Psychology, University of Western Ontario, London, Ontario N6A 5C2, Canada.

As noted by Saucier and Goldberg (1998), the Big Five factors originated with studies of the structure underlying personality ratings, ratings made using words taken from the English-language personality lexicon. When personality adjectives are sampled from the language, either exhaustively or randomly, and used to describe real people, they tend to define five orthogonal clusters. The names applied to those lexically based clusters, in decreasing order by size, are Extraversion, Agreeableness, Conscientiousness, Emotional Stability versus Neuroti- cism, and Intellect or Imagination. Roughly the same five dimensions tend to be found in the same order with impressive consistency across different samples of subjects, different selections of personality adjectives, and even different cultures and language groups. The Big Five factors of personality are thought to be important precisely because they are big. According to the Lexical Hypothesis, the words we have invented to describe individual differences in personality are reflections of real human behaviors, and the number of words we have invented is in direct proportion to the importance of the behavior domain described. As Goldberg (1982) has said, “Those individual differences that are most significant in the daily transactions of persons with each other become encoded into their language. The more important such a difference is, the more people will notice it and wish to talk of it, with the result that eventually they will invent a word for it” (p. 204). The discovery of the Big Five factors of personality in linguistic data has subsequently led to the structural evaluation of questionnaires and other personality instruments, instruments that may or may not be purposely designed to measure those factors. Sometimes, however, the lexical and nonlexical (i.e., questionnaire) factors have differed in mor- phology. For example, whereas the lexical studies typically find a factor (the smallest) called Intellect, questionnaire-based measures have tended to focus on a dimension (also the smallest) called Openness to Experience (e.g., see Costa & McCrae, 1992). Other differences in Five-Factor Models can also be noted (see reviews by Block, 1995; Digman, 1990; John, 1990; McCrae & John, 1992).

The Saucier and Goldberg (1998) Analysis Goldberg has presented, in some past publications, lists of words that can be considered characteristic or prototypical of each of the lexical Big Five, based on his extensive analyses of the factor structure underlying Beyond the Big Five 823 person descriptors in the English language (e.g., Goldberg, 1982, 1990, 1992). As Saucier and Goldberg (1998) have observed, however, some of the personality adjectives evaluated have not been closely aligned with the traditional Big Five factors in certain studies. Those authors, in fact, identified 53 such adjective clusters, each relatively homogeneous but suspected by some as being potentially outside the domain of the traditional lexical Big Five. Saucier and Goldberg (1998) modified 21 of their 53 adjective clusters to make them even more homogeneous and included all 74 groups of words in their analysis. Their intent was to evaluate the extent to which each cluster of words shows statistical allegiance to the lexical Big Five. To do this, they administered the personality adjectives along with traditional Big Five marker adjectives to subjects in a self-report format. They then correlated scores on each of the 74 word clusters with scores on each of the five factors. Their ultimate test of whether or not a particular word cluster belonged within the space of the Big Five was to compute a multiple correlation between that cluster and the five factor measures. A word cluster that has much variance in common with one or more of the Big Five would, of course, have a high multiple correlation with those factors. The 74 multiple correlations of Saucier and Goldberg’s word clusters with their Big Five marker variables ranged from .09 to .67 with a mean of .38. Using an arbitrary cutoff score of .30 to decide whether a particular word cluster was or was not within the five-factor domain, those authors identified some person descriptor clusters, mainly related to physical characteristics, as being beyond the Big Five. Those clusters, each having a multiple correlation with the factors of less than .30, included words referring to a person’s height, weight, age, and physical attractiveness. Importantly, only one personality-relevant word cluster was likewise judged not to be within the bounds of the traditional Big Five, and it was related to religiosity, containing words such as religious, devout, and reverent. With regard to religiosity, however, Saucier and Goldberg claimed that “many might class- ify this cluster as reflecting individual differences in attitudes or ideology, rather than personality” (p. 506).

Saucier and Goldberg’s Data Revisited Saucier and Goldberg (1998) have done a commendable job of identify- ing clusters of personality adjectives that appear, at least at face value, 824 Paunonen & Jackson not to have a strong relation to any of the Big Five factors. But their analysis led them to the conclusion that almost all of those personality variables are, in fact, related to the Big Five nontrivially. Upon our initial read of their article, we found this conclusion a curious one, given the nature of some of the person descriptors involved (see Saucier and Goldberg, 1998, Table 1). Consider, for example, the three dimensions masculine versus feminine, witty and humorous, cunning and sly. None of these seems to belong firmly to any Big Five factor. Saucier and Goldberg, however, concluded that all three dimensions do indeed fit into the space of the Big Five. But notice that a cluster of person descriptive adjectives related to one’s financial wealth (i.e., being prosperous, rich, and well-to-do versus being poor), by Saucier and Goldberg’s own criterion, was found to be more a part of the Big Five factors than were dimensions related to being masculine-feminine, or being witty and humorous, or being cunning and sly (1998, Table 2). Curious indeed. As we are about to describe, our consideration of Saucier and Gold- berg’s (1998) results has led us to the conclusion that many of the variables assessed in that study do not have a lot in common with the traditional lexical Big Five. But this conclusion stands in direct contrast to that of Saucier and Goldberg. The primary reason for this discrepancy is to be found in the choice of the statistical criterion for what constitutes a Big Five indicator. As we explain below, we take issue with Saucier and Goldberg’s criterion as being much too liberal. Table 2 in Saucier and Goldberg’s (1998) article shows the correlations between each of their 74 adjective clusters and each of their Big Five factor measures. Now note how those correlations are similar to factor loadings, being the correlations between variables and factors as found in a normal factor structure matrix. In fact, those correlations are properly interpreted as factor loadings in an extension analysis (see Dwyer, 1937; Gorsuch, 1983; Mosier, 1938). An extension analysis refers to the tech- nique of correlating variables with factors when the variables themselves were not part of the analysis that produced the factors, which is exactly what Saucier and Goldberg have done. Gorsuch (1983) mentioned a good reason for doing an extension analysis that is particularly pertinent to the present discussion. Extension analysis may be needed to test a hypothesis regarding the nature of a particular factor. A well-replicated factor may be hy- pothesized to be that which is measured by variable Z. To include Beyond the Big Five 825

variable Z in an exploratory factor analysis would usually allow it to help define the factor. Because this includes the possibility of capitalizing on chance, it would be better to determine variable Z’s relationship to the factor when the factor is based solely on the other variables. Then variable Z is correlated independently with the factor. (p. 237) Thus, Saucier and Goldberg (1998) were correct not to include their 74 adjective clusters in a factor analysis with Big Five marker variables. Their goal was to determine the extent to which those clusters, thought to be questionable in this context, fall in the space of the Big Five. Their goal was not to use those clusters to help determine the space itself (see also Paunonen & Jackson, 1996). As already mentioned, the extension loadings reported by Saucier and Goldberg (1998) in their Table 2 are simply factor loadings that one might see in a typical factor analysis. Such extension loadings, however, will generally be slightly lower in magnitude than will factor loadings in which the variables have been included in the factor analysis. The reason is that, with a typical variable-factor correlation, the factor contains the variable itself as part of the linear composite, inflating the correlation somewhat (this is the capitalization on chance referred to in Gorsuch’s quote above). This difference in value between an extension loading and a regular factor loading for a variable, however, should not be great if that variable is one of many that define the factor. How does one normally determine, in a regular factor analysis, whether or not a particular variable falls in the space of the common factors extracted? First, one would naturally look at the size of the largest loading for the variable to see whether or not it clearly defines a particular factor. As noted by Zwick and Velicer (1982), loadings of less than about .40, and certainly of less than .30, would hardly qualify as salient loadings. What about Saucier and Goldberg’s extension loadings? Of 74 variables, three had loadings of .50 or greater, three had loadings of .40 to .50, and 18 had loadings of .30 to .40. That leaves 50 personality word clusters with factor loadings so small in size that one could not reasonably consider them to define a particular factor of the Big Five exclusively.1

1. Saucier and Goldberg (1998) also cited Zwick and Velicer, but in defense of their choice of a .30 cutoff multiple correlation as indicating whether or not a variable falls within the space of the Big Five factors. In this context, they wrote “Zwick and Velicer (1982) noted that loadings below .30 or .40 are usually ignored in applications of principal 826 Paunonen & Jackson

Now, a variable might have no salient loading on any one factor but, instead, might split across two or more factors. Such a variable could still be considered to fall within the space of the common dimensions, representing a blend of multiple factors. But before such an interpretation is warranted, one must eliminate the possibility that the variable has projections into higher dimensionalities beyond the common factor space, and the way to do that is to calculate its communality. The communality of a variable is computed as the sum of its squared factor loadings taken across the common factors. But note that this is also the computation for the (squared) multiple correlation between the variable and the (orthogonal) factors (Harman, 1976, p. 83). In fact, what Saucier and Goldberg reported as multiple correlations in their Table 2 are no more than the unsquared communalities of the variables in their Big Five factor space. Normally one would not consider the multiple correlation between a variable and a set of factors in deciding whether that variable falls within the common factor space. Instead, one would evaluate the variable’s communality. So what do Saucier and Goldberg’s (1998) personality variables look like in that context? By squaring the multiple correlations of their 74 adjective clusters with their Big Five factors, we find variable communalities ranging from a .01 to .45 with a mean of only .16. Recall also their cutoff multiple correlation value of .30 for determining whether a variable does or does not fall within the space of the Big Five. That value corresponds to communality of only .09. We submit that a communality of .09 is not of sufficient magnitude to support the claim that the variable falls within the factor space in question. Such a variable would have only 9% of its variance being due to the factors but 91% of its variance being due to some combination of error variance and specific variance. (The error and specific variance components of a variable are collectively referred to as its uniqueness and, together with its communality, make up its total variance.) But there components analysis” (p. 514). The implication of their statement is that the variable- factors multiple correlations they computed are no different than factor loadings, and the same criteria that are used to evaluate the one can be applied to the other. As we are about to demonstrate, however, the multiple correlation of a variable with a set of factors is not the variable’s factor loading. The variable’s factor loading is its simple correlation with a single factor. And, as far as Saucier and Goldberg’s extension factor loadings are concerned (Table 2), the large majority of those simple correlations do not even come close to meeting Zwick and Velicer’s .30–.40 cutoff for defining the factors in question. Beyond the Big Five 827 is more. Knowing the communalities and reliabilities of each of Saucier and Goldberg’s 74 clusters permits us to partition those variables’ vari- ances into their separate common, specific, and error components (see Harman, 1976, p. 20). As mentioned, the proportion of total variance in common with the Big Five factors varied from 1% to 45% for the 74 variable clusters, with a mean of 16%. The specificities of those variables, in contrast, can be computed as ranging from 21% to 88% with a mean of 58%. What does this mean? It means that the uniquenesses of the variables are not all due to error variance. It means that those variables with small communalities and large specificities have substantial projections into dimensions beyond the Big Five.

Possible Outliers to the Big Five We next decided to reevaluate Saucier and Goldberg’s 74 clusters in terms of possible consistencies in personality content, but content that might be beyond the Big Five. To do this we first eliminated from consideration variables having to do with physical characteristics (e.g., short-tall), demographics (e.g., employed-unemployed), low base rate undesirable behaviors (e.g., evil, cruel), and variables otherwise not clearly related to traditional personality traits (e.g., lucky-unlucky). We also relaxed somewhat Saucier and Goldberg’s criterion of a .09 communality in deciding on whether a variable does or does not fall outside the domain of the Big Five factors. We chose a more reasonable, but still liberal, communality of .20 as our criterion. (We discuss these arbitrary cutoffs later in this article.) Our reevaluation of Saucier and Goldberg’s (1998) 74 adjective clusters led us to identify 26 of them as appearing to constitute meaningful personality dimensions that, in the present data, all had communalities of less than .20 while having high reliabilities. The mean communality of the 26 clusters was, in fact, only .13, but with a mean reliability of .72. These figures indicate 26 internally consistent clusters of variables having nontrivial projections into a hyperspace that is beyond the five dimensions evaluated by Saucier and Goldberg. Some of the 26 clusters we identified as relatively independent of the Big Five obviously overlapped with one another. In fact, they appeared to us to describe perhaps nine distinguishable bipolar dimensions of personality. We describe these dimensions below with reference to one 828 Paunonen & Jackson pole only and in no particular order. The numbers in parentheses refer to Saucier and Goldberg’s (1998) word cluster identifiers. 1. Religious, devout, reverent. This set of four adjective clusters (nos. 9, 9r, 14, 20) had the lowest communality of all the groups we identified in this section. Those four clusters had a mean communality of only .07. Saucier and Goldberg (1998) have admitted that the dimension of religiosity is probably the most likely candidate in their set of variables to reside beyond the traditional Big Five personality factors (see also Goldberg, 1990). 2. Sly, deceptive, manipulative. Six clusters of adjectives (nos. 12, 12r, 18, 18r, 21, 21r) related to this personality dimension had a mean communality of .13 in the present five-factor space. Interestingly, several studies in the past with the Social Astuteness scale of the Jackson Personality Inventory–Revised (JPI-R; Jackson, 1994) have supported the notion that this domain of behaviors does not fit well into the traditional nonlexical Big Five factor space (see Ashton, Jackson, Helmes, & Paunonen, 1998; Detwiler & Ramanaiah, 1996; Paunonen & Jackson, 1996). 3. Honest, ethical, moral. Three clusters of variables (nos. 1, 1r, 26) related to honesty had a mean communality of .11. There is substantial independent evidence that behaviors related to this dimension of personality are largely orthogonal to the traditional Big Five factors. Ashton, Lee, and Son (in press), for example, have com- piled the results of several cross-cultural studies showing that honesty might best be considered another “big” personality factor in its own right, being at least as large as the Intellect factor in many lexical studies (see also Peabody, 1987). 4. Sexy, sensual, erotic. The three clusters (nos. 7, 15, 33) defining this content dimension had a mean communality of .13. As noted by Saucier and Goldberg (1998), Buss (1996) has already suggested that variables related to this class of behaviors do not fit well within the Big Five personality factors. 5. Thrifty, frugal, miserly. Three variable clusters (nos. 4, 4r, 16) defined this dimension with a mean communality of .16. Some additional evidence for the low communality within the space of the Big Five of an adjective cluster comprised of the words Beyond the Big Five 829

economical and thrifty can be found in an article by Goldberg (1990, Table 4). 6. Conservative, traditional, down-to-earth. Four clusters of personality adjectives (nos. 10, 25, 41, 43) defining this domain had a mean communality of .15. Note that this dimension of behavior is similar to that measured by the JPI-R Traditional Values scale, a scale that has had noticeably low communalities in some past nonlexical studies of the Big Five (Ashton, Jackson, Helmes, & Paunonen, 1998; Paunonen & Jackson, 1996). This dimension has been impli- cated as a potential Big Five outlier even in other lexical studies (see John, 1990). 7. Masculine-feminine. This one cluster (no. 35) was defined by those two words alone, and had a communality of .13. Masculinity- femininity has been considered in the past as a dimension not closely aligned with the Big Five (see Noller, Law, & Comrey, 1987; Paunonen, 1993). Interestingly, Saucier and Goldberg (1998) had two other clusters of person descriptors related to masculinity and femininity (nos. 24, 24r), but they did not meet our criterion for variables lying outside of the Big Five. Those other two clusters, however, contained adjectives such as rough, coarse, and callous, giving those dimensions relatively high loadings on Saucier and Goldberg’s Agreeableness factor. 8. Egotistical, conceited, snobbish. A single cluster of adjectives (no. 46) related to egocentric behaviors had a communality of .16. Block (1995) has suggested the relative independence of this domain from the Big Five, a view that has been expressed by McCrae, Costa, and Busch (1986). 9. Humorous, witty, amusing. This one cluster of adjectives (no. 44) had a communality of only .13. Linguistic studies of the Dutch language also have suggested that this dimension of personality does not fit well within the Big Five personality factors (De Raad & Hoskens, 1990). Also, one of Goldberg’s (1990) analyses of self-ratings on the terms humorous and witty found that cluster (Table 4), which he assigned to the Extraversion factor, to have a mere .08 communality in the space of the Big Five. 830 Paunonen & Jackson

There is at least one other dimension of personality that we would add to the list above—the dimension related to risk-taking or thrill-seeking behaviors. Several nonlexical studies in the past have had difficulty in placing this domain of behaviors within the space of the Big Five (Ashton, Jackson, Helmes, & Paunonen, 1998; Jackson, Paunonen, Fraboni, & Goffin, 1996; Paunonen & Jackson, 1996; Zuckerman, Kuhl- man, Joireman, Teta, & Kraft, 1993). Curiously, that dimension was not represented in Saucier and Goldberg’s (1998) set of 74 adjective clusters. We do not intend to imply that the nine or ten dimensions described above are mutually orthogonal. For example, some of those listed (e.g., deceptiveness and ethicalness) might be subsumed by a broader and more general honesty factor (see Ashton, Lee, & Son, in press). Nor do we intend to imply that those dimensions are as well represented in the English language as are the traditional Big Five. Clearly they are not. But whether or not that fact means that those dimensions are any less important to understanding human behavior is debatable, and it is an issue we raise again at the end of this article.

On the Sizes of Communalities Saucier and Goldberg (1998) interpreted communalities of .09 or greater as indicating variables that are important facets of a particular factor space. We, instead, chose a more conservative communality criterion of .20. The difference in interpretation based on the two criteria is that, in the former case, one is led to conclude that there is little variance in human behavior beyond the Big Five factor space, whereas, in the latter case, one must conclude that there is much variance in behavior not accounted for by those factors. Is our communality criterion of .20 too conservative? We think not. Consider our 26 clusters of personality-relevant adjectives that all met this criterion, having a mean communality of .13. Twenty-four of those clusters had communalities less than did two other clusters in the analysis related to being rich or poor, the latter having a mean communality of .17. Surely, personal wealth, even with a communality of .17, should not be considered to be within the domain of the Big Five personality factors for theoretical reasons alone. And if our supposition is correct, then it follows that the other 24 behavior-based variables in the present analysis, with lower communalities, are even less a part of the Big Five. Beyond the Big Five 831

How do Saucier and Goldberg’s (1998) communalities compare to factor results reported elsewhere? In the context of typical simple structure factor analyses, even when using established Big Five marker variables, their values are very low. For example, Goldberg (1990) has presented a list of 100 adjective clusters that purport to represent the core of the Big Five factors of personality (see also Goldberg, 1992). The mean communality of those marker variables in one of his analyses was .37 (Table 4, self-ratings). Compare that to the mean communality of only .16 for the 74 clusters evaluated by Saucier and Goldberg (1998). Recall that Saucier and Goldberg (1998) established an arbitrary cutoff of .30 for the multiple correlation between a variable and the Big Five factors in deciding whether or not the variable fell within the space of the factors. But how does a multiple correlation of that size compare with a multiple correlation based on other personality variables? By their stan- dard, we can show that even some of the Big Five factors do not fall beyond the space of the other four factors! Let’s consider a study by Goldberg (1992), in which he calculated several estimates of the cross-correlations of sets of aggregated Big Five marker variables. In Table 5 of that study, he presented six Big Five factor-factor correlation matrices (based on self-ratings, peer ratings, standardized scores, unstandardized scores, etc.). Even if one takes the correlation matrix showing the most orthog- onality among the Big Five factors (based on standardized self-ratings), two of those factors are very close to meeting Saucier and Goldberg’s criterion for not falling the beyond the space of the other four: Agree- ableness has a multiple correlation of .28 with the other four factors, and Conscientiousness has a multiple correlation of .29. Our example is the best case scenario in Goldberg’s (1992) Table 5, and any of the other five correlation matrices presented there yield even higher multiple correlations, far exceeding the .30 cutoff score. Thus, by Saucier and Goldberg’s criterion, one is led to the conclusion that some of the Big Five factors of personality, purportedly mutually orthogonal, do not themselves fall beyond the space of the other four factors.2

2. Even questionnaire measures of the Big Five show such multicollinearity. Evaluation of the NEO-PI-R (Costa & McCrae, 1992, Appendix F), for example, shows that three of its Big Five Domain scales each have multiple correlations with the other four exceeding .50, one has a multiple correlation exceeding .40, and one factor almost, but not quite, reaches Saucier and Goldberg’s (1998) .30 cutoff criterion with a multiple correlation of .28. 832 Paunonen & Jackson

A Final Word We believe that there is much important variance in human behavior not accounted for by the Big Five personality factors. This variance is nonrandom and is related to internally consistent, theoretically meaningful classes of behavior that have, both historically and in Saucier and Goldberg’s (1998) own data, failed to correlate highly with traditional Big Five dimensions. One reason these domains of behavior have generally not been isolated empirically from typical Big Five lexical factors is because, as we have shown here, such variables are often forced into the five-factor space with remarkably trivial communalities (see also Saucier & Goldberg, 1996). Another reason why the behavior domains we described in this study are lacking in most lexical Big Five results is probably because they are not well represented in the language of personality. As such, they yield small factors in any analysis of linguistic structure, at least relative to salient Big Five variables, and are thus excluded from the final factor solution. But just because the words describing a domain of behavior are relatively few in number, does that mean that the domain is any less important than is some bigger one? The Big Five factors no doubt represent prominent higher-order dimensions of individual differences that have become lavishly encoded in the language. But if the language contains relatively few words to describe a dimension of behavior, does that necessarily mean it is less important in describing a person than is a larger dimension? Think of the dimension of masculinity-femininity, referred to earlier in this article. Neither word is found among Goldberg’s (1990) list of 339 trait adjectives purportedly representing the Big Five. The probable reason is that there are few words in the English language that are gramatically syno- nyms or antonyms of masculine or feminine. Masculinity-femininity, therefore, is unlikely to define its own factor in any comprehensive analysis of the structure underlying personality descriptors because that dimension, if it exists, will be a small one. As most criteria for factor extraction and rotation are based on factor size, the masculinity-femininity dimension will be excluded from the final solution. But is the excised dimension not an important one? As far as masculinity-femininity is concerned, some have viewed it as important enough to add to their personality inventories, including the Comrey Scales (Comrey, 1970), the California Psychological Inventory (Gough, 1987), and the Minne- sota Multiphasic Personality Inventory (MMPI) (Hathaway & McKinley, Beyond the Big Five 833

1967). Furthermore, evidence presented here and elsewhere (Noller, Law, & Comrey, 1987) indicates that this dimension of behavior has a substantial projection into a higher dimensionality than the five typically evaluated. Our view is that the ultimate test of whether a dimension of behavior is important to the understanding of human behavior depends not on the size of the factor in the language of personality, as the Lexical Hypothesis would have us believe. The test should instead be one of incremental utility. If one can identify theoretically meaningful, internally consistent classes of behavior that are able to predict socially and personally significant life criteria, then such personality dimensions are important (e.g., see Ashton, Jackson, Paunonen, Helmes, & Rothstein, 1995; Mershon & Gorsuch, 1988; Paunonen, 1998; Paunonen & Ashton, 2000; Paunonen, Rothstein, & Jackson, 1999; Rothstein, Paunonen, Rush, & King, 1994). Moreover, if such dimensions are able to account for criterion variance not accounted for by the Big Five personality factors, then those dimensions need to be considered separately in any comprehensive description of the determinants of human behavior.

REFERENCES Ashton, M. C., Jackson, D. N., Helmes, E., & Paunonen, S. V.(1998). Joint factor analysis of the Personality Research Form and the Jackson Personality Inventory? Journal of Research in Personality, 32, 243–250. Ashton, M. C., Jackson, D. N., Paunonen, S. V., Helmes, E., & Rothstein, M. G. (1995). The criterion validity of broad factor scales versus specific trait scales. Journal of Research in Personality, 29, 432–442. Ashton, M. C., Lee, K., & Son (in press). Honesty as the sixth factor of personality: Correlations with Machiavellianism, primary psychopathy, and social adroitness. European Journal of Personality. Block, J. (1995). A contrarian view of the Five-Factor Approach to personality description. Psychological Bulletin, 117, 187–215. Buss, D. M. (1996). Social adaptation and five major factors of personality. In J. S. Wig- gins (Ed.), Theoretical perspectives for the Five-Factor Model (pp. 180–207). New York: Guilford Press. Comrey, A. L. (1970). Manual for the Comrey Personality Scales. San Diego, CA: Educational and Industrial Testing Service. Costa, P.T., Jr., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psy- chological Assessment Resources, Inc. De Raad, B., & Hoskens, M. (1990). Personality-descriptive nouns. European Journal of Personality, 4, 131–146. 834 Paunonen & Jackson

Detwiler, F. R. J., & Ramanaiah, N. V. (1996). Structure of the Jackson Personality Inventory from the perspective of the five-factor model. Psychological Reports, 65, 947–950. Digman, J. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41, 417–440. Dwyer, P. S. (1937). The determination of factor loadings of a given test from the known factor loadings of other tests. Psychometrika, 2, 173–178. Goldberg, L. R. (1982). From ace to zombie: Some explorations in the language of personality. In C. D. Spielberger & J. N. Butcher (Eds.), Advances in personality assessment (Vol. 1, pp. 203–234). Hillsdale, NJ: Erlbaum. Goldberg, L. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 26–42. Gorsuch, R. L. (1983). Factor analysis. Hillsdale, NJ: Lawrence Erlbaum. Gough, H. G. (1987). Manual for the California Psychological Inventory. Palo Alto, CA: Consulting Psychologists Press. Harman, H. H. (1976). Modern factor analysis. Chicago: University of Chicago Press. Hathaway, S. R., & McKinley, J. C. (1967). Minnesota Multiphasic Personality Inven- tory: Manual for administration and scoring. New York: Psychological Corporation. Jackson, D. N. (1994). Jackson Personality Inventory–Revised manual. Port Huron, MI: Research Psychologists Press. Jackson, D. N., Paunonen, S. V., Fraboni, M., & Goffin, R. G. (1996). A five-factor versus a six-factor model of personality structure. Personality and Individual Differences, 20, 33–45. John, O. P. (1990). The “Big Five” factor taxonomy: Dimensions of personality in the natural language and in questionnaires. In L. A. Pervin (Ed.), Handbook of personality: Theory and research (pp. 66–100). New York: Springer-Verlag. McCrae, R. R., Costa, P.T., Jr., & Busch, C. M. (1986). Evaluating the comprehensiveness in personality systems: The California Q-set and the five factor model. Journal of Personality, 54, 430–446. McCrae, R. R., & John, O. P. (1992). An introduction to the five-factor model and its applications. Journal of Personality, 60, 175–215. Mershon, B., & Gorsuch, R. L. (1988). Number of factors in the personality sphere: Does increase in factors increase predictability of real-life criteria? Journal of Personality and Social Psychology, 55, 675–680. Mosier, C. I. (1938). A note on Dwyer: The determination of the factor loadings of a given test. Psychometrika, 3, 297–299. Noller, P., Law, H., & Comrey, A. L. (1987). Cattell, Comrey, and Eysenck personality factors compared: More evidence for the five robust factors? Journal of Personality and Social Psychology, 53, 775–782. Paunonen, S. V.(1993, August). Sense, nonsense, and the Big Five factors of personality. Paper presented at the annual meeting of the American Psychological Association, Toronto, Ontario. Paunonen, S. V. (1998). Hierarchical organization of personality and prediction of behavior. Journal of Personality and Social Psychology, 74, 538–556. Beyond the Big Five 835

Paunonen, S. V., & Ashton, M. C. (2000). Big-five predictors of academic achievement. Manuscript submitted for publication. Paunonen, S. V., & Jackson, D. N. (1996). The Jackson Personality Inventory and the Five-Factor Model of personality. Journal of Research in Personality, 30, 42–59. Paunonen, S. V., Rothstein, M. G., & Jackson, D. N. (1999). Narrow reasoning about the use of broad personality measures in personnel selection. Journal of Organizational Behavior, 20, 389–405. Peabody, D. (1987). Selecting representative trait adjectives. Journal of Personality and Social Psychology, 52, 59–71. Rothstein, M. G., Paunonen, S. V., Rush, J. C., & King, G. A. (1994). Personality and cognitive ability predictors of performance in graduate business school. Journal of Educational Psychology, 86, 516–530. Saucier, G., & Goldberg, L. R. (1996). Evidence for the Big Five in analyses of familiar English personality adjectives. European Journal of Personality, 10, 61–77. Saucier, G., & Goldberg, L. R. (1998). What is beyond the Big Five? Journal of Personality, 66, 495–524. Zuckerman, M., Kuhlman, D. M., Joireman, J., Teta, P.,& Kraft, M. (1993). A comparison of three structural models for personality: the Big Three, the Big Five, and the Alternative Five. Journal of Personality and Social Psychology, 65, 757–768. Zwick, W. R., & Velicer, W. F. (1982). Factors influencing four rules for determining the number of components to retain. Multivariate Behavioral Research, 17, 253–269.