Memory & Cognition 1998,26 (1). 146-160 The perception offace gender: The role of stimulus structure in recognition and classification ALICE J. O'TOOLE University ofTexas at Dallas, Richardson, Texas KENNETH A. DEFFENBACHER University ofNebraska, Omaha, Nebraska and DOMINIQUE VALENTIN, KAREN McKEE, DAVID HUFF,and HERvE ABDI University ofTexas at Dallas, Richardson, Texas The perception of face gender was examined in the context of extending "face space" models of human face representations to include the perceptual categories defined by male and female faces. We collected data on the recognizability, gender classifiability (reaction time to classify a face as male/fe­ male), attractiveness, and masculinity/femininity of individual male and female faces. Factor analyses applied separately to the data for male and female faces yielded the following results. First, for both male and female faces, the recognizability and gender classifiability of faces were independent-a re­ sult inconsistent with the hypothesis that both recognizability and gender classifiability depend on a face's "distance" from the subcategory gender prototype. Instead, caricatured aspects of gender (femininity/masculinity ratings) related to the gender classifiability of the faces. Second, facial attrac­ tiveness related inversely to face recognizability for male, but not for female, faces-a result that re­ solves inconsistencies in previous studies. Third, attractiveness and femininity for female faces were nearly equivalent, but attractiveness and masculinity for male faces were not equivalent. Finally,we ap­ plied principal component analysis to the pixel-coded face images with the aim of extracting measures related to the gender classifiability and recognizability of individual faces. We incorporated these model-derived measures into the factor analysis with the human rating and performance measures. This combined analysis indicated that face recognizability is related to the distinctiveness of a face with respect to its gender subcategory prototype. Additionally, the gender classifiability offaces related to at least one caricatured aspect of face gender. Human faces provide us with a plethora ofinformation tion. The importance ofvisually derived semantic informa­ that is valuable and necessary for social interaction. When tion for understanding human performance on face pro­ we encounter a face, we can quickly and efficiently decide cessing tasks has become increasingly evident in recent whether it is one we know. For faces ofpersons we know, attempts to bridge the gap between the perceptual and we can often retrieve semantic and identity information memory-based components ofthe face processing system about the person. Additionally, from both familiar and (Hancock, Burton, & Bruce, 1996; O'Toole, Abdi, Deffen­ unfamiliar faces we can make judgments about the gen­ bacher, & Valentin, 1995; O'Toole, Deffenbacher, Valen­ der, approximate age, and race ofthe person. The infor­ tin, & Abdi, 1994). mation we use to accomplish these latter judgments has In the present study, we concentrate on the categorical been referred to by Bruce and Young(1986) in their model dimension offace gender. We begin by reviewing briefly offace processing as "visually derived semantic" informa- the basic psychological findings supporting a prototype­ based, face space conceptualization of human face pro­ cessing. This abstract representational framework pro­ vides a parsimonious account ofseveral well-established This work was supported in part by NIMH Grant MH5 I765-02 to psychological findings concerning face typicality (Light, A.J.O. Thanks are due to June Chance and Al Goldstein for providing Kayra-Stuart, & Hollander, 1979; Valentine, 1991; Val­ the faces used in the present experiments and simulations and to James C. Bartlett, Shimon Edelman, Heinrich H. Biilthoff, George Wolford, entine & Bruce, 1986). We then consider the implications and two anonymous reviewers for very helpful comments and sugges­ of extending this representational framework to include tions on an earlier version of this manuscript. Correspondence should natural, perceptual categories offaces, such as face gen­ be addressed to A, J. O'Toole, School of Human Development, GR4.1, der. These categories share the configural base ofa face University of Texas at Dallas, Richardson, TX 75083-0688 (e-mail: [email protected]). prototype, but differ in the nature ofthe visually derived semantic information that specifies the subcategorical ~Accepted by previous editor, Geoffrey R. Loftus face configurations. Copyright 1998 Psychonomic Society, Inc. 146 PERCEIVING FACE GENDER 147 Face Spaces, Typicality, and Gender Categories the sample. Combined, these data suggest that although The internal representation ofa facial prototype, aver­ the "averageness" or (proto )typicality ofa face may re­ age face, or common facial configuration has played a late to its attractiveness, it is not likely to be the only de­ prominent role in many theories of human face process­ termining factor. ing and has been called variously a "facial prototype" (Val­ Despite the central importance ofthe average/prototype entine & Bruce, 1986), "CONSPEC" (Morton & John­ face to theories ofhuman face processing, little is known son, 1991), and a "face schema" (Goldstein & Chance, about how face configurations/prototypes specific to sub­ 1980). The psychological evidence supporting a face categories offaces, such as male or female faces, or faces prototype comes from the well-established relationships ofdifferent races, relate to this theoretical construct. In reported between facial ratings and human performance addition to the common configuration that all faces share, on face processing tasks. For example, it is well known there exist several subcategories offaces with somewhat that faces judged to be typical are less accurately recog­ different configural bases. These include the visually de­ nized than are faces judged to be unusual (Light et al., rived semantic subcategories associated with race, gen­ 1979). This occurs under the assumption that the theo­ der, and perhaps age.' Although faces within these sub­ retical face space is more "crowded" close to the proto­ groups share the general face configuration (i.e., the type, and so typical faces are more confusable with other relative position of eyes, nose, and mouth), different vi­ faces than are distinctive faces. Additionally, faces judged sually derived semantic subgroups can be distinguished to be typical are classified as faces more quickly than are from one another by normative and variational differences facesjudged to be unusual (Valentine& Bruce, 1986).This in (I) feature-based information, (2) "second-order" con­ result occurs under the assumption that typical faces are figural information (see Rhodes, 1988), or (3) in some closer to the prototype than are unusual faces, and so can combination ofboth. For example, faces ofdifferent races be compared to the prototype more quickly than can un­ differ in the norm and variability offeatures like eye color, usual faces (Valentine & Bruce, 1986). hair color, and eye shape, and may also differ in norms In addition to the basic findings concerning face typ­ related to general face shape such as the degree of pro­ icality, human observer ratings of facial attractiveness trusion ofthe facial features. Likewise, male and female have also been shown to vary inversely with face recog­ faces differ normatively in feature-based information such nizability (Light, Hollander, & Kayra-Stuart, 1981), in­ as the size of the nose and prominence of the brow and dicating, by implication, that attractive faces may in also in more global facial shape characteristics such as some ways be "average." Data from an earlier study by "fleshiness" (Enlow, 1982). We will refer to these norma­ Shepherd and Ellis (1973), however, are not entirely con­ tive differences between male and female faces as "stim­ sistent with the results ofLight et ai. (1981). Shepherd and ulus structure differences." Ellis examined the effects ofattractiveness (high, medium, and low) on recognizability at three delay periods (1, 6, Stimulus Structure Differences Between and 35 days). In the short and intermediate delay condi­ Male and Female Faces tions, they found no effects of attractiveness on recog­ Psychological studies. Questions concerning the na­ nizability. In the 35-day delay condition, however, they ture of stimulus structure differences between male and found a U-shaped relationship between attractiveness female faces can be considered both from a psychologi­ and recognizability, with very attractive and very unattrac­ cal and a computational perspective. From a psycholog­ tive faces better recognized than moderately attractive ical perspective, in recent years there has been an intense faces. An important difference between the study ofLight interest in determining the information human observers et al. and that ofShepherd and Ellis, however, is the gen­ use to determine the gender ofa face (e.g., Brown & Per­ der of faces used as stimuli; Light et al. used only male rett, 1993; Bruce et aI., 1993; Bruce & Langton, 1994; faces, whereas Shepherd and Ellis used only female faces. Burton, Bruce, & Dench, 1993; Chronicle et aI., 1995; The relationship between perceived attractiveness and Roberts & Bruce, 1988; Yamaguchi, Hirukawa, & Kana­ computationally defined facial averages has been de­ zawa, 1995). These researchers have measured or manip­ bated vigorously in