Unraveling the Distributed Neural Code of Facial Identity Through Spatiotemporal Pattern Analysis
Total Page:16
File Type:pdf, Size:1020Kb
Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis Adrian Nestor1, David C. Plaut, and Marlene Behrmann Department of Psychology, Carnegie Mellon University, and Center for the Neural Basis of Cognition, Pittsburgh, PA 15213 Edited by Charles G. Gross, Princeton University, Princeton, NJ, and approved May 12, 2011 (received for review February 13, 2011) Face individuation is one of the most impressive achievements of Furthermore, insofar as face individuation is mediated by a our visual system, and yet uncovering the neural mechanisms network, it is important to determine how information is distrib- subserving this feat appears to elude traditional approaches to uted across the system. Some interesting clues come from the fact functional brain data analysis. The present study investigates the that right fusiform areas are sensitive to both low-level properties neural code of facial identity perception with the aim of ascertain- of faces (16, 25) and high-level factors (26, 27), suggesting that ing its distributed nature and informational basis. To this end, these areas may mediate between image-based and conceptual we use a sequence of multivariate pattern analyses applied to representations. If true, such an organization should be reflected in functional magnetic resonance imaging (fMRI) data. First, we the pattern of information sharing among different regions. combine information-based brain mapping and dynamic discrimi- The current work investigates the nature and the extent of nation analysis to locate spatiotemporal patterns that support face identity-specific neural patterns in the human ventral cortex. We classification at the individual level. This analysis reveals a network examined functional MRI (fMRI) data acquired during face of fusiform and anterior temporal areas that carry information individuation and assessed the discriminability of activation about facial identity and provides evidence that the fusiform face patterns evoked by different facial identities across variation in area responds with distinct patterns of activation to different face expression. To uncover the neural correlate of identity recogni- identities. Second, we assess the information structure of the tion, we performed dynamic multivariate mapping by combining fi network using recursive feature elimination. We nd that diagnos- information-based mapping (28) and dynamic discrimination tic information is distributed evenly among anterior regions of the analysis (29). The results revealed a network of fusiform and NEUROSCIENCE mapped network and that a right anterior region of the fusiform anterior temporal regions that respond with distinct spatiotem- gyrus plays a central role within the information network mediat- poral patterns to different identities. To elucidate the distribu- fi ing face individuation. These ndings serve to map out and tion of information, we examined the distribution of diagnostic characterize a cortical system responsible for individuation. More fi information across these regions using recursive feature elimi- generally, in the context of functionally de ned networks, they nation (RFE) (30) and related the information content of dif- provide an account of distributed processing grounded in ferent regions to each other. We found that information is evenly information-based architectures. distributed among anterior regions and that a right fusiform region plays a central role within this network. he neural basis of face perception is the focus of extensive re- Tsearch as it provides key insights both into the computational Results architecture of visual recognition (1, 2) and into the functional Participants performed an individuation task with faces (Fig. 1) organization of the brain (3). A central theme of this research and orthographic forms (OFs) (Fig. S1). Specifically, they recog- emphasizes the distribution of face processing across a network of nized stimuli at the individual level across image changes in- spatially segregated areas (4–10). However, there remains consid- troduced by expression (for faces) or font (for OFs). Response erable disagreement about how information is represented and accuracy was at ceiling (>95%) as expected given the familiar- processed within this network to support tasks such as individuation, ization with the stimuli before scanning and the slow rate of expression analysis, or high-level semantic processing. stimulus presentation. Thus, behavior exhibits the expected invari- One influential view proposes an architecture that maps dif- ance to image changes, and the current investigation focuses on ferent tasks to distinct, unique cortical regions (6) and, as such, the neural codes subserving this invariance. draws attention to the specificity of this mapping (11–20). As a case in point, face individuation (e.g., differentiating Steve Jobs Dynamic Multivariate Mapping. The analysis used a searchlight (SL) from Bill Gates across changes in expression) is commonly map- with a 5-voxel radius and a 3-TR (Time to Repeat) temporal ped onto the fusiform face area (FFA) (6, 21). Although recent envelope to constrain spatiotemporal patterns locally. These patterns were submitted to multivariate classification on the basis studies have questioned this role of the FFA (14, 15), overall they Methods SI Text agree with this task-based architecture as they single out other of facial identity ( and ). The outcome of the areas supporting individuation. analysis is a group information-based map (28) revealing the strength of discrimination (Fig. 2). Each voxel in this map rep- However, various distributed accounts have also been consid- fi ered. One such account ascribes facial identity processing to resents an entire region of neighboring voxels de ned by the SL mask. multiple, independent regions. Along these lines, the FFA’s sen- sitivity to individuation has been variedly extended to areas of the inferior occipital gyrus (5), the superior temporal sulcus (12), Author contributions: A.N., D.C.P., and M.B. designed research; A.N. performed research; and the temporal pole (22). An alternative scenario is that identity A.N., D.C.P., and M.B. analyzed data; and A.N., D.C.P., and M.B. wrote the paper. is encoded by a network of regions rather than by any of its separate The authors declare no conflict of interest. — components such a system was recently described for subordinate- This article is a PNAS Direct Submission. level face discrimination (23). Still another distributed account Data deposition: The fMRI dataset has been deposited with the XNAT Central database attributes individuation to an extensive ventral cortical area rather under the project name “STSL.” than to a network of smaller separate regions (24). Clearly, the 1To whom correspondence should be addressed. E-mail: [email protected]. degree of distribution of the information supporting face individ- This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. uation remains to be determined. 1073/pnas.1102433108/-/DCSupplemental. www.pnas.org/cgi/doi/10.1073/pnas.1102433108 PNAS Early Edition | 1of6 Downloaded by guest on September 29, 2021 Fig. 2. Group information-based map of face individuation. The map is computed using a searchlight (SL) approach and estimates the discrimina- < Fig. 1. Experimental face stimuli (4 identities × 4 expressions). Stimuli were bility of facial identities across expression (q 0.05). Each voxel in the map fi matched with respect to low-level properties (e.g., mean luminance), ex- represents the center of an SL-de ned region supporting identity discrimi- ternal features (hair), and high-level characteristics (e.g., sex). Face images nation. The four slices show the sensitivity peaks of the four clusters revealed courtesy of the Face-Place Face Database Project (http://www.face-place.org/) by this analysis. Copyright 2008, Michael J. Tarr. Funding provided by NSF Award 0339122. The results above suggest that identity coding relies on a dis- Our mapping revealed four areas sensitive to individuation: two tributed cortical system. Clarifying the specificity of this system located bilaterally in the anterior fusiform gyrus (aFG), one in the to face individuation is addressed by our region-of-interest right anterior medial temporal gyrus (aMTG), and one in the left (ROI) analyses. posterior fusiform gyrus (pFG). The largest of these areas cor- responded to the right (r)aFG whereas the smallest corresponded ROI Analyses. First, we examined whether the FFA supports reli- fi to its left homolog (Table 1). able face individuation as tested with pattern classi cation. Bi- fi To test the robustness of our mapping, the same cortical volume lateral FFAs were identi ed in each subject using a standard face (Fig. S2A) was examined with SL masks of different sizes. These localizer, and discrimination was computed across all features in — alternative explorations produced qualitatively similar results a region given the use of spatiotemporal patterns, our features X (Fig. S3 A–C). In contrast, a univariate version of the mapping (SI are voxel time-point pairings rather than voxels alone. The Text fi analysis revealed above-chance performance for rFFA (Fig. 3). ) failed to uncover any signi cant regions, even at a liberal fi threshold (q < 0.10), attesting to the strength of multivariate; To reduce over tting, we repeated the analysis above using subsets of diagnostic features identified by multivariate feature mapping. fi To further evaluate these results,