Supplementary materials for: High-level cognition during story listening is reflected in high-order dynamic correlations in neural activity patterns

1 1,2 1, Lucy L. W. Owen , Thomas H. Chang , and Jeremy R. Manning † 1Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 2Amazon.com, Seattle, WA †Address correspondence to [email protected] September 9, 2019

1 Order

0 5 10 15

Order 1 Order 2 Order 3 Order 4 Order 5

speech sts v1 anterior insula visual sts voice early visual insula occipital auditory temporal sulcus cuneus stop signal cuneus voice sounds visual cortex stop ofc superior temporal speech visual anterior cortex ofc sounds comprehension blind unpleasant visual cortex stg tom occipital insular cortex visual eld auditory cortex network vision response orbitofrontal heschl superior temporal sighted response inhibition angular temporal person primary visual vlpfc extrastriate

Order 6 Order 7 Order 8 Order 9 Order 10

temporal lobe hippocampus recognition monitoring insular lobe hippocampal temporal lobes anterior insula intensity amygdala memory encoding fusiform gyrus conict posterior insula putamen lobe mtl fusiform face insula anterior insular cortex ligual temporal lobe anterior temporal frontal cortex secondary somatosensory categories mtl face a medial frontal tactile pole medial temporal fusiform insula distriminative insula anterior hippocampus a anterior insula disease ad encoding temporal lobe error sii temporal pole amygdala categories insular primary secondary

Order 11 Order 12 Order 13 Order 14 Order 15

hippocampal retrosplenial parahippocampal cortex ba index nger hippocampus temporal pole objects ba 44 contralateral memory navigation parahippocampal extrastriate nger anterior hippocampus face recognition place pleasant motor task episodic anterior temporal episodic memory faces hand amygdala pole episodic fusiform s1 medial temporal recognize locations face sensorimotor cortex retrieval angular memories object primary somatosensory parahippocampal angularl gyrus medial temporal object recognition m1 mtl cuneus autobiographical viewing ipsilateral

Figure S1: Top terms associated with the endpoints of the strongest correlations for the intact experimental condition. Each color corresponds to one order of inter-subject functional correlations. The inflated brain plots display the locations of the endpoints of the 10 strongest (absolute value) correlations at each order, projected onto the cortical surface (Combrisson et al., 2019). The lists of terms display the top 10 Neurosynth terms (Rubin et al., 2017) decoded from the corresponding brain maps for each order. (Also see Fig. 7, top row, in the main text.)

2 Order

0 5 10 15

Order 1 Order 2 Order 3 Order 4 Order 5

auditory sounds sounds decision task occipital sounds auditory auditory visual stimulus inferior occipital speech speech superior temporal recruited dlpfc auditory cortex superior temporal pitch word pairs visual superior temporal voice speech regardless thalamic stg stg planum respectively lateral occipital voice auditory cortex auditory cortex generally cortex dlpfc heschl planum stg stimulus extrastriate pitch spoken temporale classic dorsomedial heschl gyrus sts planum temporale character dorsolateral prefrontal

Order 6 Order 7 Order 8 Order 9 Order 10

putamen inferior frontal face recognition nucleus accumbens prefrontal music lobe mtl verb accumbens dorsolateral prefrontal vocal medial temporal face subgenual precuneus posterior mind tom frontal gyrus frontal gyrus ventral striatum prefrontal cortex tom parahippocampal cortex a prediction cortex precuneus auditory stimuli semantic inferior frontal dorsomedial perfronal dorsomedial heard dorsolateral prefrontal fusiform face prediction error dorsolateral theory of mind mtl recognition outcome medial mind lateral prefrontal verbs dorsomedial precuneus tone concepts face a ventral premotor posteral cingulate

Order 11 Order 12 Order 13 Order 14 Order 15

2 somatosensory orthographic inferior parietal anterior superior sts touch reading decision speech temporal sulcus somatosensory cortex chinese ips auditory sulcus sts parietal junction visual word choice listening mental states tactile form sulcus ips spoken dorsomedial pain word form anterior intraparietal frontal operculum dorsomedial prefrontal junction word maintenance temporal gyrus speaker 1 somatosensory occipitotemporal intraparietal superior temporal medial somatosensory written intraparietal sulcus thalamic superior temporal s1 words decision making sounds superior

Figure S2: Top terms associated with the endpoints of the strongest correlations for the paragraph exper- imental condition. This figure is in the same format as Figure S1, but displays results for the paragraph- scrambled story listening condition. (Also see Fig. 7, second row, in the main text.)

3 Order

0 5 10 15

Order 1 Order 2 Order 3 Order 4 Order 5

speech speech sts semantic brainstem sounds sts speech temporal lobe pain auditory voice voice fronto temporal spoken voice auditory superior temporal meaning fronto temporal superior temporal superior temporal auditory sentences speech perception stg sounds sounds syntactic speech sts stg stg verb repetition auditory cortex auditory cortex temporal familiarity insular acoustic audiovisual listening comprehension language comprehension temporal temporal acoustic language anterior superior

Order 6 Order 7 Order 8 Order 9 Order 10

autobiographical anterior temporal memory retrieval fusiform gyrus objects posterior cingulate amygdala episodic fusiform fusiform default fearful retrieval face fusiform gyrus pcc fear episodic memory fusiform face extrastriate default mode amygdala insula recollection objects object medial amygdala responses retrieved face a occipitotemporal supremarginal amygdala response encoding a visual eld vmpfc neutral thalamic occipitotemporal visual mpfc fearful faces lobe mtl faces lateral occipital supramarginal gyrus neutral faces memories category perception

Order 11 Order 12 Order 13 Order 14 Order 15

ventral premotor medial prefrontal retrosplenial object medial premotor cortex medial retrosplenial cortex objects posterior cingulate premotor mpfc precuneus face recognition pcc tactile cortex mpfc brainstem lateral occipital medial prefrontal sensorimotor prefrontal cortex posterior cingulate precuneus corte precuneus somatosensory prefrontal personal tools autobiographical anterior intraparietal ipl medial mt default motor default medial prefrontal v5 default mode inferior parietal autobiographical mpfc motion lateral parietal production default mode place action observation mpfc

Figure S3: Top terms associated with the endpoints of the strongest correlations for the word experimental condition. This figure is in the same format as Figure S1, but displays results for the word-scrambled story listening condition. (Also see Fig. 7, third row, in the main text.)

4 Order

0 5 10 15

Order 1 Order 2 Order 3 Order 4 Order 5

parietal visual network autobiographical dorsolateral calculation occipital expectancy anterior temporal dorsolateral prefrontal ips v1 task default network cognitive intraparietal extrastriate ica autobiographic dlpfc superior parietal motion additional default memory load tasks sighted networks real tasks subtraction visual cortex load scene working task object memory load mental working memory action occiplical cortex spl retrosplenial lateral prefrontal numbers visual eld parietal dmn load

Order 6 Order 7 Order 8 Order 9 Order 10

dorsomedial anterior insula occipital posterior insula person medial insula precuneus amygdala hippocampus face a dorsomedial prefrontal frontal cortex precuneus posterior accumbens fusiform face cortex mpfc anterior visual nucleus accumbens dorsolateral prefrontal medial prefrontal monitoring cortex precuneus amygdala a mpfc insular parietal anterior hippocampus production memories insula anterior ipl hippocampus categories temporal pole picture occipital cortex hippocampal theory mind pole error superior parietal monetary tpj recollection conict motion sii speech

Order 11 Order 12 Order 13 Order 14 Order 15

1 somatosensory orthographic olfactory amygdala response nucleus accumbens hippocampal reading amygdala amygdala hippocampus monetary incentive hippocampus chinese taste amygdala responses prediction error somatosensory cortex visual word insula retrosplenial accumbens tactile form disgust episodic incentive s1 word form ratings retrieval striatum amygdala hippocampus word amygdala response amygdala reinforcement somatosensory occipitotemporal painful hippocampus monetary touch written food emotional stimuli ventral striatum sensory word posterior insula fearful prediction

Figure S4: Top terms associated with the endpoints of the strongest correlations for the rest experimental condition. This figure is in the same format as Figure S1, but displays results for the resting state condition. (Also see Fig. 7, bottom row, in the main text.)

5 Supplemental references

Combrisson, E., Vallat, R., O’Reilly, C., Jas, M., Pascarella, A., l Saive, A., Thiery, T., Meunier, D., Altukhov, D., Lajnef, T., Ruby, P., Guillot, A., and Jerbi, K. (2019). Visbrain: a multi- purpose GPU-accelerated open-source suite for multimodal brain data visualization. Frontiers in Neuroinformatics, 13(14):1–14.

Rubin, T. N., Kyoejo, O., Gorgolewski, K. J., Jones, M. N., Poldrack, R. A., and Yarkoni, T. (2017). Decoding brain activity using a large-scale probabilistic functional-anatomical atlas of human cognition. PLoS Computational Biology, 13(10):e1005649.

6