Research Articles: Behavioral/Cognitive A heteromodal word-meaning binding site in the visual word form area under top-down frontoparietal control https://doi.org/10.1523/JNEUROSCI.2771-20.2021 Cite as: J. Neurosci 2021; 10.1523/JNEUROSCI.2771-20.2021 Received: 30 October 2020 Revised: 19 February 2021 Accepted: 22 February 2021 This Early Release article has been peer-reviewed and accepted, but has not been through the composition and copyediting processes. The final version may differ slightly in style or formatting and will contain links to any extended data. Alerts: Sign up at www.jneurosci.org/alerts to receive customized email alerts when the fully formatted version of this article is published. Copyright © 2021 the authors 1 A heteromodal word-meaning binding site in the visual word form area 2 under top-down frontoparietal control 3 Abbreviated title: A general word-meaning binding site within VWFA 4 5 Lang Qin1,2*, Bingjiang Lyu3*, Su Shu1,4, Yayan Yin5,6, Xiongfei Wang7,8, 6 Jianqiao Ge1, Wai-Ting Siok2, Jia-Hong Gao1,4, 9† 7 1 Center for MRI Research, Academy for Advanced Interdisciplinary Studies, 8 Peking University, Beijing, China 9 2 Department of Linguistics, the University of Hong Kong, Hong Kong, China 10 3 Centre for Speech, Language and the Brain, Department of Psychology, University of Cambridge, 11 UK, CB2 3EB 12 4 Beijing City Key Lab for Medical Physics and Engineering, Institute of Heavy Ion Physics, School 13 of Physics, Peking University, Beijing, China 14 5 Department of Radiology, Xuanwu Hospital, Capital Medical University, Beijing, China 15 6 Beijing Key Laboratory of Magnetic Resonance Imaging and Brain Informatics, Beijing, China 16 7 Department of Neurosurgery, Sanbo Brain Hospital, Capital Medical University, Beijing, China 17 8 Beijing Key Laboratory of Epilepsy, Beijing, China 18 9 McGovern Institute for Brain Research, Peking University, Beijing, China 19 * Lang Qin and Bingjiang Lyu contributed equally to this work. 20 † Correspondence should be addressed to Jia-Hong Gao, Center for MRI Research, Academy for 21 Advanced Interdisciplinary Studies, Peking University, Beijing, China. Email: [email protected]. 22 23 Number of pages: 62 24 Number of figures: 5 25 Number of tables: 12 26 Number of words for abstract: 226 27 Number of significance statement:112 28 Number of words for introduction: 649 29 Number of words for discussion: 1406 30 Conflicts of interest statement: The authors declare no competing financial interests. 31 Acknowledgements: This work was supported by the National Natural Scientific Foundation of China 32 (Grants 81790650, 81790651, 81727808, 31421003, 81627901, and 31771253), the National Key 33 Research and Development Program of China (2017YFC0108900), Beijing Municipal Science & 34 Technology Commission (Grant Z181100001518003) and Capital's Funds for Health Improvement and 35 Research (2020-4-801). We thank the National Centre for Protein Sciences at Peking University in Beijing, 36 China, for assistance with data acquisition and analyses. 1 37 Abstract: The integral capacity of human language together with semantic memory drives 38 the linkage of words and their meaning, which theoretically is subject to cognitive control. 39 However, it remains unknown whether, across different language modalities and input/output 40 formats, there is a shared system in the human brain for word-meaning binding and how this 41 system interacts with cognitive control. Here, we conducted a functional magnetic resonance 42 imaging experiment based on a large cohort of subjects (50 females, 50 males) to 43 comprehensively measure the brain responses evoked by semantic processing in spoken and 44 written word comprehension and production tasks (listening, speaking, reading and writing). 45 We found that heteromodal word input and output tasks involved distributed brain regions 46 within a frontal-parietal-temporal network and focally coactivated the anterior lateral visual 47 word form area (VWFA), which is located in the basal occipitotemporal area. Directed 48 connectivity analysis revealed that the VWFA was invariably under significant top-down 49 modulation of the frontoparietal control network and interacts with regions related to 50 attention and semantic representation. This study reveals that the VWFA is a key site 51 subserving general semantic processes linking words and meaning, challenging the 52 predominant emphasis on this area’s specific role in reading or other general visual 53 processes. Our findings also suggest that the dynamics between semantic memory and 54 cognitive control mechanisms during word processing are largely independent of the 55 modalities of input or output. 56 Keywords: controlled semantic cognition; word-meaning binding; visual word form area; 57 word comprehension and production; fMRI 2 58 Significance statement: Binding words and their meaning into a coherent whole during 59 retrieval requires accessing semantic memory and cognitive control, allowing our thoughts to 60 be expressed and comprehended through mind-external tokens in multiple modalities, such as 61 written or spoken forms. However, it is still unknown whether multimodal language 62 comprehension and production share a common word-meaning binding system in human 63 brains and how this system is connected to a cognitive control mechanism. By systematically 64 measuring brain activity evoked by spoken and written verbal input and output tasks tagging 65 word-meaning binding processes, we demonstrate a general word-meaning binding site 66 within the visual word form area and how this site is modulated by the frontal-parietal control 67 network. 3 68 Introduction 69 Binding words and their meaning as a central language processing is dependent on 70 semantic memory, wherein concepts are extracted from or externalized into forms in different 71 modalities, such as spoken or written words. Semantic memory comprises conceptual 72 associations abstracted from experiences without reference to specific instances, including 73 word meanings, objects and facts(Martin, 2006; Patterson et al., 2007; Binder and Desai, 74 2011; Forseth et al., 2018). When in the service of language, multiple lead-in processes 75 access the semantic memory repository and drive word-meaning binding to enable verbal 76 comprehension and production(Indefrey and Levelt, 2004; Forseth et al., 2018), which should 77 be modulated by cognitive control(Binder and Desai, 2011). Nevertheless, it is unclear (i) 78 where the modality-invariant word-meaning binding system is located in the brain and (ii) 79 how this system is subject to semantic control. 80 Attempts have been made to identify the neural correlates underlying semantic 81 representational systems across input modalities. A strong resemblance has been 82 demonstrated between two sets of semantic representations recalled by spoken and written 83 stories during language comprehension(Deniz et al., 2019), and the left anterodorsal pars 84 triangularis (Brodmann area 45, BA45) is significantly correlated with cross-modal semantic 85 similarity encoding(Liuzzi et al., 2017). Furthermore, previous literature suggests that word 86 storage and retrieval mainly rely on ventrolateral temporal lobes, particularly the middle 87 temporal gyrus (MTG), inferior temporal gyrus (ITG) and fusiform gyrus (FG)(Binder et al., 88 2009; Davis and Gaskell, 2009). The ventrolateral temporal lobes include several regions for 89 phonology-semantics-orthography linkage, including the lexical interface(Hickok and 90 Poeppel, 2007; Forseth et al., 2018), the left ventral anterior temporal lobe (ATLv) (also 91 referred to as basal temporal language area, BTLA)(Binney et al., 2010; Purcell et al., 2014) 92 and the visual word form area (VWFA)(Dehaene and Cohen, 2007, 2011). These anatomical 4 93 and functional areas have long been considered heteromodal integration regions, and recent 94 findings corroborate that posterior parts of the lateral MTG, ITG and middle FG are centrally 95 involved in representing lexical items independently of modalities (Forseth et al., 2018; 96 Evans et al., 2019; Mattioni et al., 2020). Latest studies with epilepsy patients that used 97 naming tasks to tag the word processing uncovered that the middle occipitotemporal cortex 98 (i.e., FG and ITG) may function as lexical semantic hub and play a crucial part in associating 99 words and their meaning(Forseth et al., 2018; Binder et al., 2020). 100 Recent research proposed controlled semantic cognition (CSC) framework wherein 101 semantic cognition is dependent on two principal interacting neural systems: one for semantic 102 representation and another for controlling the activity within the representational system 103 according to specific contexts(Lambon Ralph et al., 2017). According to the CSC, concepts 104 arise from multidimensional verbal and nonverbal experiences encoded in modality-specific 105 brain regions and the multimodal hub situated in the bilateral ATLv. The ATLv necessarily 106 mediates the interactions among modality-specific attributes (Patterson et al., 2007; Lambon 107 Ralph et al., 2017) and is engaged across different modalities(verbal and nonverbal 108 semantics, auditory and visual input presentation)(Visser et al., 2012; Rice et al., 2015). Prior 109 research on cognitive control in general(O'Reilly et al., 2002; Badre et al., 2005; Badre and 110 D'esposito, 2009; Cole et al., 2013), semantic retrieval specifically(Thompson-Schill et al., 111 1997; Wagner et al., 2001) and disordered semantic