Sensitivity to Faces with Typical and Atypical Part Configurations Within Regions of the Face-Processing Network: an Fmri Study

Sensitivity to Faces with Typical and Atypical Part Configurations within Regions of the Face-processing Network: An fMRI Study

Andrew D. Engell1, Na Yeon Kim2, and Gregory McCarthy3

Abstract ■ Perception of faces has been shown to engage a domain- internal parts were presented in a typical configuration (two eyes specific set of brain regions, including the occipital face area above a nose above a mouth) or in an atypical configuration (the (OFA) and the fusiform face area (FFA). It is commonly held that locations of individual parts were shuffled within the face outline). the OFA is responsible for the detection of faces in the environ- Perception of the atypical faces evoked a significantly larger re- ment, whereas the FFA is responsible for processing the identity sponse than typical faces in the OFA and in a wide swath of the of the face. However, an alternative model posits that the FFA is surrounding posterior occipitotemporal cortices. Surprisingly, responsible for face detection and subsequently recruits the OFA typical faces did not evoke a significantly larger response than to analyze the face parts in the service of identification. An essen- atypical faces anywhere in the brain, including the FFA (although tial prediction of the former model is that the OFA is not sensitive some subthreshold differences were observed). We propose that to the arrangement of internal face parts. In the current fMRI face processing in the FFA results in inhibitory sculpting of acti- study, we test the sensitivity of the OFA and FFA to the configu- vation in the OFA, which accounts for this region’sweakerre- ration of face parts. Participants were shown faces in which the sponse to typical than to atypical configurations. ■

INTRODUCTION terior OFA to the relatively anterior FFA (Zhen, Fang, & Face perception is perhaps our most remarkable visual Liu, 2013; Liu, Harris, & Kanwisher, 2010; Caldara & ability and has therefore been the topic of intense study Seghier, 2009; Haxby et al., 2000). Consistent with this, for several decades. This research has identified many Liu and colleagues (2010) found that the OFA response brain regions that are central to the processing of faces. is not modulated by the first-order configuration of Principal among them is a region of the fusiform gyrus face parts (i.e., the arrangement of internal features), (Puce, Allison, Gore, & McCarthy, 1995; Allison, McCarthy, whereas the FFA responds more strongly to faces with Nobre, Puce, & Belger, 1994; Haxby et al., 1994; Sergent, a typical configuration (two eyes above a nose above a Ohta, & MacDonald, 1992), often referred to as the fusi- mouth). form face area (FFA; Kanwisher, McDermott, & Chun, However, this intuitively appealing hierarchical model 1997), and a region of the posterior occipitotemporal is challenged by studies of patients with unilateral or sulcus/inferior occipital gyrus, often referred to as the oc- bilateral lesions in the OFA region. Despite the absence cipital face area (OFA; Gauthier et al., 2000). However, the of an intact OFA, fMRI of these individuals shows near- functional role for each of these regions is still uncertain. typical activation of the FFA during face viewing (Weiner Cognitive models of face perception (e.g., Bruce & et al., 2016; Rossion, Dricot, Goebel, & Busigny, 2011; Young, 1986) include a node tasked with the initial struc- Schiltz et al., 2006; Steeves et al., 2006; Rossion, Caldara, tural encoding of faces that supports face detection and a et al., 2003). Additional challenges come from studies separate node tasked with a detailed analysis of the rela- that implicate the OFA in higher-order aspects of face tionship of the inner face features that supports face perception, such as identity recognition (Andrews, Davies- identification. Popular neural models (e.g., Calder & Thompson, Kingstone, & Young, 2010; Rossion, Schiltz, & Young, 2005; Haxby, Hoffman, & Gobbini, 2000) have as- Crommelinck, 2003). Puce, Allison, and McCarthy (1999) cribed these processes to the OFA and the FFA, respec- reported a patient in whom direct cortical stimulation of tively. The models posit a feed-forward and hierarchical the OFA region caused difficulty in naming a face. Similar architecture such that information flows from the pos- effects have been reported more recently, which also implicate the OFA in processing the first-order configuration of face parts (Jonas et al., 2012, 2014). Jonas and colleagues 1Kenyon College, 2Princeton University, 3Yale University (2012) reported that a patient whose descriptions of face

© 2018 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 30:7, pp. 963–972 doi:10.1162/jocn_a_01255 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 perception during OFA stimulation included “the facial elements were mixed” and “the facial elements were in disarray” (p. 285). To accommodate these results, several nonhierarchical models have been proposed that do not position the OFA as the critical “gate-keeper” into the face processing network (Duchaine & Yovel, 2015; Pitcher, Duchaine, & Walsh, 2014; Atkinson & Adolphs, 2011; Rossion, 2008; but see Yang, Susilo, & Duchaine, 2016). For instance, Rossion (2008) has put forward a model in which the FFA receives input directly from early visual cortex. Thus, the OFA is not necessary for the detection of a face but is Figure 1. Example stimuli. Examples of the different face part subsequently engaged by the FFA for “fine-grained” anal- configurations: typical and atypical. ysis of the face. An essential feature of the hierarchical model is that the OFA is not sensitive to first-order configuration, con- gories: faces with a typical first-order configuration (two sistent with its proposed role as the first node in the net- eyes above a nose above a mouth) and faces with an atyp- work tasked solely with detecting and processing face ical first-order configuration (internal features were shuf- parts. However, there is surprisingly little evidence to fled within the face; Figure 1). We used seven different support first-order configuration invariance in the OFA configurations of the left eye (L), right eye (R), mouth (but see Zhang, Li, Song, & Liu, 2012; Liu et al., 2010) (M), and nose (N) to create the atypical faces. The arrange- and there are some reports that directly contradict this ments, starting at the upper left and moving clockwise to notion (Jonas et al., 2012, 2014). Given these contradic- the upper right, lower right, and lower left, were as follows: tions, we sought to reexamine the relative sensitivities of N-R-L-M, M-R-N-L, R-N-M-L, R-M-L-N, N-R-M-L, R-M-N-L, the OFA and the FFA to the first-order configuration of M-R-L-N. In repeated versions of the same configuration, face parts in a larger sample than used in the imaging the distance between parts could vary. For instance, two studies cited above. In this fMRI study, participants faces that shared the R-M-N-L configuration might differ viewed faces in which the internal parts either had a typical in that the mouth was further offset from the right eye in or atypical first-order configuration. Of particular impor- one of the faces. With regard to placement of the eyes, tance is the response of the OFA, as the hierarchical model there were two rules that all configurations adhered predicts that this response should not be modulated as a to. First, the two eyes never appeared next to each other function of these conditions. The results will thus inform in the upper or lower half of the face. Second, an eye in and constrain neural models of face perception. the upper half of the face was always placed on the wrong side (e.g., a right eye would be placed in the upper left position). METHODS Participants Experimental Procedure Twenty-one healthy adults (10 women, mean age = In each of four runs, participants passively viewed 120 face 24.0 years, 20 right-handed) with normal vision and no images (30 from each of the four stimulus categories). history of neurological or psychiatric illness participated in Images were presented for 300 msec with a 2000-msec the study. Twenty participants identified as “white” and “ ” intertrial interval. The event-related design was jittered by one identified as Asian. All participants provided writ- including 25 “null” trials in which no image appeared ten and informed consent. The protocol was approved during the 300-msec presentation window. A unique by the Yale University institutional review board. randomized presentation order of images and nulls was created for each of the four runs. The order of runs was counterbalanced across participants. Stimuli At the conclusion of the main task, participants com- Sixty frontal-view neutral faces were created using FaceGen pleted four runs of a face localizer task. The localizer software (Singular Inversions, Toronto, Ontario, Canada). task followed the same procedure as used in Kim, Lee, An equal number of Black and White faces were included. Erlendsdottir, and McCarthy (2014). Briefly, participants All faces were male, hairless, and wore no accessories. viewed 12-sec stimulus blocks interleaved with 12-sec Faces were sized to a common height and converted to fixation blocks. Each stimulus block included eight exem- line-drawn images using GIMP software’s edge detection plar images from one of three possible categories: faces, filter (www.gimp.org). The luminance and contrast of all bodies, or houses. Participants were instructed to count faces were equalized to make low-level features as similar the number of times they saw the same picture twice in a as possible. The main task consisted of two stimulus cate- row. The localizer data collected for the current study

964 Journal of Cognitive Neuroscience Volume 30, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 were among the data used for the creation of the Atlas of We identified all local maxima with a separation dis- Social Agent Perception (ASAP; Engell & McCarthy, tance of 20 mm using AFNI’s 3dExtrema program. 2013). ROI Analyses fMRI Acquisition and Preprocessing Several ROI analyses further investigated localized differ- Brain images were acquired at the Magnetic Resonance ences between the response to typical and atypical faces. Research Center at Yale University using a Siemens 3.0-T The first ROI analysis most directly addressed our pri- TIM Trio scanner with a 12-channel head coil. Functional mary aim by evaluating the responses evoked within the images were acquired using an echo-planar pulse sequence OFA and the FFA. These regions were defined using the (repetition time [TR] = 2 sec, echo time [TE] = 25 msec, ASAP (Engell & McCarthy, 2013) and data from an inde- flip angle = 90°, matrix = 642,fieldofview[FOV]=224mm, pendent localizer task that was run at the conclusion of slice thickness = 3.5 mm, 37 slices). Two sets of structural the main experiment. Spheres (r = 17 mm) were created images were acquired for registration: coplanar images, around the OFA (48, −76, −6) and FFA (44, −48, −22) acquired using a T1 Flash sequence (TR = 300 msec, TE = voxel with highest probability of being face sensitive as 2.47 msec, flip angle = 60°, FOV = 224 mm, matrix = 2562, indicated by the ASAP. For each participant, we identified slice thickness = 3.5 mm, 37 slices), and high-resolution all clusters within each sphere at which the face > house images acquired using a 3-D MPRAGE sequence (TR = contrast yielded a z score of ≥1.65. Of these voxels, the 2530 msec, TE = 2.4 msec, flip angle = 7°, FOV = 256 mm, largest cluster was selected for the analysis. One partici- matrix = 2562, slice thickness = 1 mm, 176 slices). pant was excluded because the face localizer did not Data preprocessing and modeling were performed yield any significant voxels within either of the ROIs. usingtheFMRIBSoftwareLibrary(FSL;www.fmrib.ox. The voxels within this cluster for each participant were ac.uk/fsl). ROI and cluster analyses were performed using averagedwithineachROI:OFAandFFA.Thesemean AFNI (Analysis of Functional Neuroimages; Cox, 1996). All ROI responses from each participant were then analyzed images were skull-stripped using FSL’s brain extraction as described below. This approach allowed the ROI sizes tool. The first three volumes (6 sec) of each functional to vary across participants, thus resulting in the possibil- data set were discarded to diminish MR equilibration itythattheaccuracyoftheestimateofthemeanalso effects. Data were spatially realigned to correct for head varied accordingly. However, this should not meaning- motion using FSL’s MCFLIRT realignment tool and spa- fully affect the current results because this is a repeated- tially smoothed with a 5-mm FWHM isotropic Gaussian measures design, which means the quality of the estimates kernel. Each time series was high-pass filtered (0.02 Hz would be homogenous across conditions within each cutoff ) to eliminate low-frequency drift. Functional im- participant. ages were registered to structural images using FSL’slinear The percent signal change (PSC) of the hemodynamic registration tool (FLIRT). Functional images were first response to Black and White typical and atypical faces registered to coplanar anatomical images (6 DOF), which was averaged across all voxels within the peak cluster were then registered to high-resolution anatomical images within each ROI for each participant. Visual inspection (6 DOF), and then normalized to the Montreal Neurologi- of the time courses (PSC at each TR from −4secpre- cal Institute’s MNI 152 template (12 DOF). stimulus to 12 sec poststimulus) revealed that the maximal (or near maximal) response was, with few exceptions, at 4 sec poststimulus across all conditions. We therefore Univariate Analysis performed all subsequent ROI analyses on data from this Whole-brain voxel-wise regression analyses were per- time point. Thus, there were 160 data points for this formed using FSL’s fMRI expert analysis tool. Within each analysis: Each of the 20 participants contributed an average preprocessed run, the typical and atypical conditions were response from within both ROIs for each face type and modeled with a boxcar function convolved within a single- race. These data were analyzed with a three-way repeated- gamma hemodynamic response function. Participant-level measures ANOVA with within-subject factors of ROI (OFA, analyses (i.e., across runs) were performed using a fixed- FFA), Face configuration (typical, atypical), and Face race effects model. (black, white). Group analysis was performed using a mixed-effects model with the random effects component of variance Spatial Distribution of Response estimated using FSL’s FLAME Stage 1 + 2 procedure. For both the regression and group analyses, clusters of active In a final analysis, we investigated the anterior–posterior and voxels were identified using FSL’s two-stage procedure to medial–lateral spatial distributions of first-order configura- correct for multiple comparisons. Voxels were first thresh- tion effects within the right ventral occipitotemporal corti- olded at a level of z ≥ 2.3. A Gaussian random field theory- ces (VOTC). For this analysis, we defined two additional based cluster correction ( p < .05) was then applied to ROIs in which the average peak response was calculated correct for multiple comparisons. within voxels of individual slices along the anterior–posterior

Engell, Kim, and McCarthy 965 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 or medial–lateral axes. The difference between atypical in the ASAP. The mean PSC from each participant was and typical was calculated at each slice for each partici- calculated at 16 equidistant slices along the medial–lateral pant. The group differences at each slice were evaluated axis (MNI 22–52). tmax Permutation tests indicated that with a nonparametric tmax permutation test that avoids in- p < .05 required tcritical = ±3.01 and tcritical = ±2.83 for flation of the family-wise error rate due to multiple compar- the anterior and posterior segments, respectively. isons. t tests were run on 10,000 random permutations of the participant difference values and slice positions. A null distribution of t values was created that included the max- RESULTS imum t value from each permutation. A new critical t value Univariate Analyses was then determined by selecting the smallest value of the most extreme 5% of the distribution. At each slice, Perception of faces with an atypical arrangement of inter- Cohen’s d was used to evaluate effect size. nal features evoked significantly greater activation than The first of the spatial distribution ROIs focused on the faces with a typical arrangement throughout a large swath response within face-selective voxels of the right VOTC. of bilateral occipitotemporal cortex (Figure 2). Large clus- We created an anatomical mask of VOTC by combining ters that extended from ventral temporal to lateral occip- “ ” “ ital to superior parietal cortices were observed in both the temporal occipital fusiform cortex (TOFC) and occip- 3 3 ” – the right (64,544 mm ) and left (59,144 mm )hemi- ital fusiform gyrus areas as delineated by the Harvard 3 Oxford Structural Atlas, then further restricted the ROI to spheres. A third cluster (4264 mm )wasobservedin include only those voxels within the VOTC that were likely the right frontal pole. Within these clusters we identified, to be face selective using a probabilistic atlas for face nine, five, and two local maxima, respectively (see Table 1 perception, which is included in the ASAP (Engell & and Figure 2). McCarthy, 2013). Voxels within the anatomically defined Using a threshold of p < .05 (corrected), no significant VOTC that had a face-selective probability of p > .25 were activation was observed for the contrast of typical greater included in the ROI analysis. The mean PSC from each than atypical. participant was calculated at 26 equidistant slices along the anterior–posterior axis (MNI −38 to −88). t max ROI Analyses Permutation tests (see above) indicated that p <.05 required tcritical = ±3.05. A three-way repeated-measures ANOVA was conducted The second of the spatial distribution ROIs was de- that examined the effect of ROI, Face configuration, and signed to be less conservative and thus to explore any Race on the PSC. There was a significant interaction, F(1, 2 broader effect of first-order configuration in regions such 19) = 11.13, p =.003,ηp = .369, between the effects of as the collateral sulcus that neighbor typical “face-selective” ROI and Face configuration on the PSC (Figure 3). Paired regions. This ROI consisted of the anatomically defined samples t tests showed that this interaction is driven by a right TOFC without any additional functional constraints significantly larger response to atypical than typical faces (i.e., no minimum probability of face selectivity). The in the OFA (t = −2.59, p = .018, d = 0.91), but no differ- TOFC mask includes the collateral sulcus and inferior tem- ence in the FFA ( p =.114;seeFigure3).However, poral gyrus medial and lateral to the mid-fusiform gyrus, though not significant, we observe that, in the FFA, the respectively. The ROI was subdivided into anterior (MNI response to typical faces (M =0.054,SD = 0.036) was y: −38 to −48) and posterior (MNI y: −48 to −58) sections. numerically larger than to atypical faces (M =0.033,SD = The MNI y coordinate of −48 was chosen as the demar- 0.036). There was also a marginally significant interaction, 2 cation point between the anterior and posterior ROIs as F(1, 19) = 4.32, p =.051,ηp = .185, between the effects this is the coordinate of peak probability for face selectivity of ROI and Race on the PSC. Black faces evoked a larger

Figure 2. GLM results for the atypical > typical contrast. Left: Axial slices of the MNI 152 template brain overlaid with voxels that show a significantly (corrected for multiple comparisons) greater response to atypical than typical faces. Right: Same activation on a white matter surface. Red indicates the presence of an above threshold voxel. Yellow indicates the location of local maxima (see Univariate Analysis section).

966 Journal of Cognitive Neuroscience Volume 30, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 Table 1. Local Maxima than atypical faces at x = 40 (Figure 5). As with the typical > atypical responses found in the ROI described Right Hemisphere Left Hemisphere above, despite a relatively large effect size (d = 0.70), this X Y Z z Score X Y Z z Score difference was only significant when evaluated with the uncorrected paired samples t test. Four of the five most 36 −86 12 4.91 −34 −74 −8 4.63 medial slices showed a larger response to atypical than to 22 −64 54 4.84 −28 −78 22 4.53 typical faces with an average effect size of d =0.87(range – 56 −60 −6 4.58 −28 −58 52 4.17 0.69 1.01). But again, this difference was only significant when evaluated without multiple comparisons correction. − − − − 26 70 12 4.43 6 68 62 3.28 Within the posterior segment of this mask, the response 42 −38 42 3.67 −30 −48 −8 2.85 to atypical faces was significantly larger (average d = – 52 −56 34 3.56 1.17, range 0.88 1.33) than the response to typical faces at six adjacent medial slices; x = 34, 32, 30, 28, 26, 24. 28 62 −8 3.55 − 32 80 46 3.33 DISCUSSION 46 −40 62 3.23 The results of our study show that occipitotemporal corti- 26 −46 −14 2.96 ces (including the OFA, but also extending well beyond this 20 48 −20 2.38 region) show a strong preference for faces with atypical first-order configurations of internal features. However, MNI coordinates of local maxima identified from the group-level GLM ostensibly downstream regions of the face-processing net- atypical > typical contrast maps. Each voxel is reported with its associated z statistic. work (including the FFA) showed no significant preference to faces with typical first-order configurations of internal features. In the following discussion, we consider the im- response in the FFA than white faces, whereas black faces portance of these results and their implications for existing evoked a smaller response in the OFA than white faces. models of face perception. This marginal interaction with race is beyond the scope of this report and will not be discussed further. Why Does the OFA Prefer Atypical Feature Configurations? Spatial Distribution of Response We found that the OFA is sensitive to the first-order con- Moving from posterior to anterior within the right VOTC figuration of face parts. Most intriguing is our finding that mask, we observed that the response to atypical faces the OFA responded more strongly to atypically config- moderately increased until it plateaued across a span ured faces. We propose that this counterintuitive result from slices y = −74 to y = −64 and then monotonically decreased throughout all anterior slices (Figure 4). The response to typical faces did not vary appreciably over much of the ROI but did steadily increase across the last six most anterior slices. A significant difference ( p < .05, corrected) between the response to atypical and typical faces was observed at four adjacent slices, y = −72 to y = −66, within the maximum response plateau of the atypical faces (see above). The average effect size across these slices was d =1.07(range1.03–1.17). The response to typical faces was numerically larger than the response to atypical faces at the four most anterior slices of the ROI. These slices are within the region tradition- ally found to be the most face selective along the fusiform gyrus. When the responses were contrasted using independent paired samples t tests (i.e., uncorrected for multiple comparisons), the two most anterior slices, y = −40 and −38, were significant with marginally large effect sizes (d = 0.76 and 0.79, respectively). However, these differences were not significant in the t permu- max Figure 3. Functionally defined ROI results. Each bean plot displays the tation test that controlled for multiple comparisons. individual participant results (black lines), the distribution density of Moving from lateral to medial within the right anterior the results (typical faces in blue, atypical faces in purple), and the mean TOFC mask, we observed a greater response to typical response (red line) for the OFA and the FFA ROIs.

Engell, Kim, and McCarthy 967 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 Figure 4. PSC along the anterior–posterior axis. ROI including probable face-selective voxels of right ventral temporal cortex (see Spatial Distribution of Response section for details.) PSC evoked by the typical (cyan) and atypical (purple) face configurations was extracted from the ROI within successive 2-mm slices moving from posterior ( y = −88) to anterior ( y = −38). Error bars indicate SEM. Asterisks indicate the locations at which there was a significant difference (orange) between typical and atypical faces. Double asterisks indicate where this difference was significant after correcting for multiple comparisons

using a tmax permutation approach.

Figure 5. PSC along the medial–lateral axis. ROIs including a posterior ( y = −58 to y = −48; yellow) and anterior ( y = −48 to y = −38; red) segment (see Spatial Distribution of Response section for details). PSC evoked by the typical (cyan) and atypical (purple) face configurations was extracted from each ROI within successive 2-mm slices moving from lateral (x = 52) to medial (x =22). Error bars indicate SEM.Asterisks indicate the locations at which there was a significant difference (orange) between typical and atypical faces. Double asterisks indicate where this difference was significant after correcting for multiple comparisons using

a tmax permutation approach.

968 Journal of Cognitive Neuroscience Volume 30, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 is possibly due to inhibitory feedback from the FFA dur- to mechanisms that are not face specific. For instance, ing perception of typically configured faces. because of their inherent atypicality, these stimuli might Inhibitory feedback from the FFA to the OFA is in stimulate greater attentional engagement or visual explo- agreement with the suggestion that it is the FFA, not ration. However, prior studies of face inversion (e.g., Yovel the OFA, that is tasked with initially detecting faces in the & Kanwisher, 2005) have found no difference between environment (Rossion, Hanseeuw, & Dricot, 2012; Rossion upright and inverted faces within the OFA or an increased et al., 2011; Taubert, Apthorp, Aagten-Murphy, & Alais, response as a function of inversion that was specific to faces 2011; Jacques & Rossion, 2006, 2009; Rossion, 2008; Tsao (Watson, Huis in’t Veld, & de Gelder, 2016). Therefore, any & Livingstone, 2008). In this model, the FFA detects faces effect of face inversion is likely due to the disruption of based on holistic perception of the typical first-order con- holistic processing of faces and not mere novelty. We figuration of face parts. Following detection, the FFA en- would argue the same is likely true of the current results. gages the OFA for a more detailed processing of features Finally, we note that this finding contradicts the prior that facilitates the identification of individual faces. Thus, report of Liu and colleagues (2010). The reason for this the FFA essentially focuses the OFA on featural processing. discrepancy is not clear, though there are several differ- In the absence of such input, as would be expected in the ences between the studies. The prior work manipulated case where atypical part configuration precludes holistic two additional factors: the presence or absence of face processing in the FFA, activation in the OFA would be un- parts (the latter condition replaced parts with Black focused and domain-general and therefore relatively en- ovals) and the presence or absence of the face contour. hanced. In other words, processing of the whole in the The authors report that the presence or absence of face FFA might result in inhibitory sculpting of the processing parts did not affect the finding of configuration sensitivity of the parts in posterior occipitotemporal regions, includ- in the FFA, or the configuration insensitivity in the OFA. ing the OFA. The fact that the observed activation is widely It is unclear, however, whether the same is true of the distributed well beyond the OFA might reflect that this face contour manipulation. In the current work, the face inhibitory sculpting input from the FFA extends through- contour was always present, which makes a direct com- out the visual system. That is, rather than being initiated parison with the prior results challenging. It is also the by face detection in the OFA, it is inhibition from the case that the current study would be more sensitive to FFA that directs the visual system from general object pro- an effect given the larger sample size (ROI analysis in cessing to face-selective processing. current report, n = 20; prior report, n = 9). Recently, Nemrodov, Anderson, Preston, and Itier (2014) introduced the Lateral Inhibition, Face Template and Eye Hierarchical, Nonhierarchical, and Detector-based (LIFTED) model of holistic and featural Reciprocal Processing processing, which emphasizes lateral inhibition of features in the parafovea onto those at the fovea. They propose that A widely held theory of face processing posits that occi- this inhibition prevents overrepresentation of the foveated pitotemporal regions, including the OFA, serve as the feature (due to cortical magnification), which would pre- first node in a neural face-processing network (Zhen vent perception of the face as a whole. In their words, “This et al., 2013; Calder & Young, 2005; Haxby et al., 2000). inhibition mechanism allows features to be perceptually In this model, the OFA detects face parts and routes this ‘glued’ into a holistic representation.” (Nemrodov et al., visual information to specialized neural systems rather 2014, p. 91) Our hypothesis is broadly consistent with than general object recognition areas. This specialized the LIFTED model in that we also posit an inhibitory mech- machinery includes the FFA, which is thought to be sen- anism to account for a weaker response to holistic than sitive to the first- and second-order configuration of face part-based processing. The primary difference between parts (the distance between each part). This hierarchical the proposals is that the LIFTED model emphasizes inhibi- model posits that information flows from the OFA to the tion as a function of retinal location, whereas we emphasize FFA (Zhen et al., 2013; Liu et al., 2010; Caldara & Seghier, sculpting inhibition from the FFA to the OFA. However, 2009; Haxby et al., 2000). these processes need not be mutually exclusive. It should As noted above, an alternative nonhierarchical model also be noted that the LIFTED model is built on a large suggests that face information is first processed in the body of ERP research, whereas the current study uses fMRI. FFA and then (or possibly in parallel) in the OFA (Rossion Although the high temporal resolution of ERPs and the 2008). In this model, the FFA detects faces in the environ- high spatial resolution of fMRI are complimentary, they ment and then passes this information to the OFA for do not necessarily reflect the same underlying neural ac- more detailed processing of the individual features to tivity (cf. Engell, Huettel, & McCarthy, 2012). Additional facilitate recognition of individual faces. multimodal studies are necessary to discover how, if at The current data do not directly speak to the temporal all, these proposed inhibitory mechanisms interact and order of processing within and across the OFA and FFA. how the ERP findings integrate with those of fMRI. However, we can tentatively and speculatively infer the We also note that it is possible the significant and wide- order of processing based on the response properties spread preference for atypically configured faces is due of each region. The lack of strong preference for typically

Engell, Kim, and McCarthy 969 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 configured faces in the FFA suggests that face informa- obligatory response during face perception, whereas the tion reaches this region regardless of the configuration latter is sensitive to bottom–up (e.g., line-drawn vs. realistic of internal features and thus is not dependent on prior faces) and top–down (e.g., attention) influences. Previously, processing in the OFA. This is consistent with typical ac- single-cell recording in the macaque (Sugase, Yamane, tivation of the FFA despite a lesioned OFA (Weiner et al., Ueno, & Kawano, 1999) showed that, within the same 2016; Rossion et al., 2011; Schiltz et al., 2006; Steeves neural population, initial information processing facilitates et al., 2006; Rossion, Caldara, et al., 2003) and fMRI evi- detection of faces whereas a later wave of information dence that the FFA processes whole faces and face parts processing facilitates identification of individual faces. (Harris & Aguirre, 2008, 2010; Yovel & Kanwisher, Similarly, Ghuman and colleagues (2014) used multivariate 2004). The latter is in agreement with our recent electro- pattern classification of intracranial EEG data to investigate corticography (ECoG) report of eye-selective regions the nature of information processing in the FFA. They along the fusiform gyrus (Engell & McCarthy, 2014). foundthattheFFAwasinvolvedinfacedetectionatearly Strikingly, these eye-selective sites were more abundant latencies and face identification at longer latencies. Together, and showed greater selectivity than the face-selective sites. these studies support the notion that the stages of face The notion that the FFA is a “downstream” node in the processing might evolve within a region(s) rather than face-processing network is, in part, based on its anterior being properties of independent regions. location within the ventral visual processing stream relative to the more posterior OFA. It is inferred from these Why Does the FFA Not Prefer Typically anatomical locations that information would reach the Configured Faces? FFA at longer latencies than the OFA. This inference is indirectly supported by behavioral deficits in face pro- In three different analyses—mass univariate, ROI, and cessing induced by TMS of the OFA at latencies of 60– spatial distribution—we did not find a significant modu- 100 msec (Pitcher, Walsh, Yovel, & Duchaine, 2007). The lation of the FFA response by first-order configuration. relatively early latency of this effect is consistent with Whole-brain univariate analysis did not find any voxels the notion that the OFA is an “entry node” and precedes in which there was a stronger response to typical than the FFA. But the early disruption of the OFA does not to atypical faces. ROI analysis of the functional defined preclude the possibility of parallel—or even earlier— FFA also failed to find a significant difference, though processing in the FFA, a region that is inaccessible to the response to typical faces was numerically larger than TMS. Yue, Cassidy, Devaney, Holt, and Tootell (2011) cite to atypical faces (see Figure 3). We were surprised by this monkey data that suggest as few as one or two intervening null effect and therefore followed up the functional ROI steps as information travels from V1 to FFA (Rajimehr, analysis with an exploratory analysis of signal changes Young, & Tootell, 2009; Distler, Boussaoud, Desimone, throughout the region. This analysis also failed to find a & Ungerleider, 1993; Nakamura, Gattass, Desimone, & significantly stronger response to typical than to atypical Ungerleider, 1993; Van Essen, Anderson, & Felleman, faces. However, at a location in the neighborhood of the 1992), which allows for the possibility that initial face FFA, we did find a significant difference of typical > atyp- information reaches the FFA at very short latencies. ical when using a liberal paired samples t test at each slice Indeed, a recent ECoG study found FFA activity as early that did not correct for the family-wise error rate (16 in- as 50 msec could be used to discriminate between face dependent t tests were performed within two of the ROIs and nonface stimuli (Ghuman et al., 2014). Finally, though and 26 independent t tests were performed within the based on informal and unpublished observations from third ROI). We therefore conclude that if the FFA prefers our laboratories, comparison of face-selective ERPs evoked typical to atypically configured faces, the effect is modest. at electrodes implanted in regions of the FFA and the OFA As is the case with our finding in the OFA, the null effect (cf. Engell & McCarthy, 2011) have not shown systematic in the FFA is in contrast to a prior report that used a sim- latency differences. ilar paradigm and found a preference for typical faces in The current results and the literature cited above are this region (Liu et al., 2010). Insufficient sensitivity to de- also in line with a perspective that deemphasizes the tect such a difference in the current report is possible but order of processing altogether and instead emphasizes not likely, given its larger sample size. It is also possible reciprocal information flow (e.g., Harris & Aguirre, 2010; that the response in the FFA is disproportionally driven Schiltz, Dricot, Goebel, & Rossion, 2010) between the by the upright head/neck/chin/ears outline, which was densely connected OFA and FFA (Kim et al., 2006). This the same for both our typical and atypical faces. Though framework supports functional roles that evolve over time again, this seem unlikely, given the importance of the within a region, rather than strictly independent functional internal features of the face (Andrews et al., 2010). roles across regions. We have recently published a series One might argue that the null effect in the FFA does of ECoG studies (Engell & McCarthy, 2010, 2011, 2014) not support our speculative proposal of inhibitory feed- that report functional differences between the evoked back from the FFA to the OFA in that the feedback signal face-N200 and the induced gamma response at the same would be associated with an increased BOLD signal in the intracranial electrode sites. The former is ostensibly an former. Although this is certainly possible, we would note

970 Journal of Cognitive Neuroscience Volume 30, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 that the BOLD response to whole faces and to parts Engell, A. D., Huettel, S., & McCarthy, G. (2012). The fMRI might reflect the activation of functionally independent, BOLD signal tracks electrophysiological spectral perturbations, not event-related potentials. Neuroimage, 59, 2600–2606. but spatially comingled, neurons. Thus, the overall level Engell, A. D., & McCarthy, G. (2010). Selective attention of activation would not necessarily change, but this will modulates face-specific induced gamma oscillations recorded need to be tested by future studies. from ventral occipitotemporal cortex. Journal of Neuroscience, An alternative interpretation of the current results is that 30, 8780–8786. the presence of a face outline and of face parts, regardless Engell, A. D., & McCarthy, G. (2011). The relationship of gamma oscillations and face-specific ERPs recorded subdurally from of configuration, is sufficient for the FFA to engage the occipitotemporal cortex. Cerebral Cortex, 21, 1213–1221. OFA, thus accounting for the configuration invariance Engell, A. D., & McCarthy, G. (2013). Probabilistic atlases for within the FFA. In this account, the increased activation face and biological motion perception: An analysis of their within the OFA and throughout visual cortex is driven by reliability and overlap. Neuroimage, 74, 140–151. the increased processing demands of the atypically con- Engell, A. D., & McCarthy, G. (2014). Face, eye, and body selective responses in fusiform gyrus and adjacent cortex: An intracranial figured faces. That is, our inhibitory sculpting proposal EEG study. Frontiers in Human Neuroscience, 8, 642. accounts for the net activation difference within the OFA Gauthier, I., Tarr, M. J., Moylan, J., Skudlarski, P., Gore, J. C., & as being driven by an inhibitory signal from the FFA during Anderson, A. W. (2000). The fusiform face area is part of a perception of typical configurations; the alternative “men- network that processes faces at the individual level. Journal – tal sweat” interpretation would account for this same net of Cognitive Neuroscience, 12, 495 504. Ghuman, A. S., Brunet, N. M., Li, Y., Konecky, R. O., Pyles, J. A., activation difference as being driven by increased local Walls, S. A., et al. (2014). Dynamic encoding of face information processing demands within the OFA and visual cortex in the human fusiform gyrus. Nature Communications, 5, during perception of atypical configurations. 5672. Harris, A., & Aguirre, G. K. (2008). The representation of parts and wholes in face-selective cortex. Journal of Cognitive Acknowledgments Neuroscience, 20, 863–878. Harris, A., & Aguirre, G. K. (2010). Neural tuning for face wholes The authors thank Mr. William Walker for his help in data col- and parts in human fusiform gyrus revealed by fMRI adaptation. lection. This research was supported by National Institute of Journal of Neurophysiology, 104, 336–345. Mental Health grant MH-005286 (G. M.). Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The Reprint requests should be sent to Andrew D. Engell, Depart- distributed human neural system for face perception. Trends ment of Psychology, Kenyon College, Samuel Mather Hall, in Cognitive Sciences, 4, 223–233. Gambier, OH 43022, or via e-mail: [email protected]. Haxby, J. V., Horwitz, B., Ungerleider, L. G., Maisog, J. M., Pietrini, P., & Grady, C. L. (1994). The functional organization of human extrastriate cortex: A PET-rCBF study of selective REFERENCES attention to faces and locations. Journal of Neuroscience, 14, 6336–6353. Allison, T., McCarthy, G., Nobre, A., Puce, A., & Belger, A. Jacques, C., & Rossion, B. (2006). The speed of individual face (1994). Human extrastriate visual cortex and the perception categorization. Psychological Science, 17, 485–492. of faces, words, numbers, and colors. Cerebral Cortex, 4, Jacques, C., & Rossion, B. (2009). The initial representation of 544–554. individual faces in the right occipito-temporal cortex is Andrews, T. J., Davies-Thompson, J., Kingstone, A., & Young, holistic: Electrophysiological evidence from the composite A. W. (2010). Internal and external features of the face are face illusion. Journal of Vision, 9, 1–16. represented holistically in face-selective regions of visual Jonas, J., Descoins, M., Koessler, L., Colnat-Coulbois, S., Sauvée, cortex. Journal of Neuroscience, 30, 3544–3552. M., Guye, M., et al. (2012). Focal electrical intracerebral Atkinson, A. P., & Adolphs, R. (2011). The neuropsychology of stimulation of a face-sensitive area causes transient face perception: Beyond simple dissociations and functional prosopagnosia. Neuroscience, 222, 281–288. selectivity. Philosophical Transactions of the Royal Society Jonas, J., Rossion, B., Krieg, J., Koessler, L., Colnat-Coulbois, of London, Series B, Biological Sciences, 366, 1726–1738. S., Vespignani, H., et al. (2014). Intracerebral electrical Bruce, V., & Young, A. (1986). Understanding face recognition. stimulation of a face-selective area in the right inferior occipital British Journal of Psychology, 77, 305–327. cortex impairs individual face discrimination. Neuroimage, Caldara, R., & Seghier, M. L. (2009). The fusiform face area 99, 487–497. responds automatically to statistical regularities optimal for Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The face categorization. Human Brain Mapping, 30, 1615–1625. fusiform face area: A module in human extrastriate cortex Calder, A. J., & Young, A. W. (2005). Understanding the specialized for face perception. Journal of Neuroscience, 17, recognition of facial identity and facial expression. Nature 4302–4311. Reviews Neuroscience, 6, 641–651. Kim, M., Ducros, M., Carlson, T., Ronen, I., He, S., Ugurbil, K., Cox, R. W. (1996). AFNI: Software for analysis and visualization et al. (2006). Anatomical correlates of the functional organization of functional magnetic resonance neuroimages. Computers in the human occipitotemporal cortex. Magnetic Resonance and Biomedical Research, 29, 162–173. Imaging, 24, 583–590. Distler, C., Boussaoud, D., Desimone, R., & Ungerleider, L. G. Kim, N. Y., Lee, S. M., Erlendsdottir, M. C., & McCarthy, G. (1993). Cortical connections of inferior temporal area TEO (2014). Discriminable spatial patterns of activation for faces in macaque monkeys. Journal of Comparative Neurology, and bodies in the fusiform gyrus. Frontiers in Human 334, 125–150. Neuroscience, 8, 632. Duchaine, B., & Yovel, G. (2015). A revised neural framework Liu, J., Harris, A., & Kanwisher, N. (2010). Perception of face for face processing. Annual Review of Vision Science, 1, parts and face configurations: An fMRI study. Journal of 393–416. Cognitive Neuroscience, 22, 203–211.

Engell, Kim, and McCarthy 971 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021 Nakamura, H., Gattass, R., Desimone, R., & Ungerleider, L. G. Schiltz, C., Sorger, B., Caldara, R., Ahmed, F., Mayer, E., Goebel, (1993). The modular organization of projections from areas R., et al. (2006). Impaired face discrimination in acquired V1 and V2 to areas V4 and TEO in macaques. Journal of prosopagnosia is associated with abnormal response to Neuroscience, 13, 3681–3691. individual faces in the right middle fusiform gyrus. Cerebral Nemrodov, D., Anderson, T., Preston, F. F., & Itier, R. J. (2014). Cortex, 16, 574–586. Early sensitivity for eyes within faces: A new neuronal account Sergent, J., Ohta, S., & MacDonald, B. (1992). Functional of holistic and featural processing. Neuroimage, 97, 81–94. neuroanatomy of face and object processing. A positron Pitcher, D., Duchaine, B., & Walsh, V. (2014). Combined TMS emission tomography study. Brain, 115, 15–36. and fMRI reveal dissociable cortical pathways for dynamic and Steeves, J. K., Culham, J. C., Duchaine, B. C., Pratesi, C. C., static face perception. Current Biology, 24, 2066–2070. Valyear, K. F., Schindler, I., et al. (2006). The fusiform face Pitcher, D., Walsh, V., Yovel, G., & Duchaine, B. (2007). TMS area is not sufficient for face recognition: Evidence from a evidence for the involvement of the right occipital face area patient with dense prosopagnosia and no occipital face area. in early face processing. Current Biology, 17, 1568–1573. Neuropsychologia, 44, 594–609. Puce, A., Allison, T., Gore, J. C., & McCarthy, G. (1995). Face- Sugase, Y., Yamane, S., Ueno, S., & Kawano, K. (1999). Global sensitive regions in human extrastriate cortex studied by and fine information coded by single neurons in the temporal functional MRI. Journal of Neurophysiology, 74, 1192–1199. visual cortex. Nature, 400, 869–873. Puce, A., Allison, T., & McCarthy, G. (1999). Taubert, J., Apthorp, D., Aagten-Murphy, D., & Alais, D. (2011). Electrophysiological studies of human face perception. III: The role of holistic processing in face perception: Evidence Effects of top–down processing on face-specific potentials. from the face inversion effect. Vision Research, 51, 1273–1278. Cerebral Cortex, 9, 445–458. Tsao, D. Y., & Livingstone, M. S. (2008). Mechanisms of face Rajimehr, R., Young, J. C., & Tootell, R. B. (2009). An anterior perception. Annual Reviews Neuroscience, 31, 411–437. temporal face patch in human cortex, predicted by macaque Van Essen, D. C., Anderson, C. H., & Felleman, D. J. (1992). maps. Proceedings of the National Academy of Sciences, Information processing in the primate visual system: An U.S.A., 106, 1995–2000. integrated systems perspective. Science, 255, 419–423. Rossion, B. (2008). Constraining the cortical face network by Watson, R., Huis In ’t Veld, E. M., & de Gelder, B. (2016). neuroimaging studies of acquired prosopagnosia. The neural basis of individual face and object perception. Neuroimage, 40, 423–426. Frontiers in Human Neuroscience, 10, 66. Rossion, B., Caldara, R., Seghier, M., Schuller, A. M., Lazeyras, Weiner, K. S., Jonas, J., Gomez, J., Maillard, L., Brissart, H., F., & Mayer, E. (2003). A network of occipito-temporal face- Hossu, G., et al. (2016). The face-processing network is sensitive areas besides the right middle fusiform gyrus is resilient to focal resection of human visual cortex. Journal of necessary for normal face processing. Brain, 126, 2381–2395. Neuroscience, 36, 8425–8440. Rossion, B., Dricot, L., Goebel, R., & Busigny, T. (2011). Yang, H., Susilo, T., & Duchaine, B. (2016). The anterior Holistic face categorization in higher order visual areas of temporal face area contains invariant representations of face the normal and prosopagnosic brain: Toward a non-hierarchical identity that can persist despite the loss of right FFA and view of face perception. Frontiers in Human Neuroscience, OFA. Cerebral Cortex, 26, 1096–1107. 4, 225. Yovel, G., & Kanwisher, N. (2004). Face perception: Domain Rossion, B., Hanseeuw, B., & Dricot, L. (2012). Defining specific, not process specific. Neuron, 44, 889–898. face perception areas in the human brain: A large-scale Yovel, G., & Kanwisher, N. (2005). The neural basis of the factorial fMRI face localizer analysis. Brain and Cognition, behavioral face-inversion effect. Current Biology, 15, 2256–2262. 79, 138–157. Yue, X., Cassidy, B. S., Devaney, K. J., Holt, D. J., & Tootell, R. B. Rossion, B., Schiltz, C., & Crommelinck, M. (2003). The (2011). Lower-level stimulus features strongly influence functionally defined right occipital and fusiform. Neuroimage, responses in the fusiform face area. Cerebral Cortex, 21, 35–47. 19, 877–883. Zhang, J., Li, X., Song, Y., & Liu, J. (2012). The fusiform face area Schiltz, C., Dricot, L., Goebel, R., & Rossion, B. (2010). Holistic is engaged in holistic, not parts-based, representation of perception of individual faces in the right middle fusiform faces. PLoS One, 7, e40390. gyrus as evidenced by the composite face illusion. Journal of Zhen, Z., Fang, H., & Liu, J. (2013). The hierarchical brain Vision, 10, 1–16. network for face recognition. PLoS One, 8, e59886.

972 Journal of Cognitive Neuroscience Volume 30, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01255 by guest on 26 September 2021