Neural Processing of Facial Expressions As Modulators of Communicative Intention Facial Expression in Communicative Intention

Neural processing of facial expressions as modulators of communicative intention Facial expression in communicative intention

Rasgado-Toledo Jalil1, Valles-Capetillo Elizabeth1, Giudicessi Averi2, Giordano Magda1*.

1Departament of Behavioral and Cognitive Neurobiology, Instituto de Neurobiología, Universidad Nacional Autónoma de México, Blvd. Juriquilla 3001, Juriquilla, Querétaro, 76230 México

2 Department of Psychiatry, University of California, San Diego 9500 Gilman Drive La Jolla, CA * Corresponding author:

E-mail address: [email protected] (Giordano, Magda)

Number of pages: 31 Number of figures: 8 Number of tables: 1 Number of words: Abstract: 250 Introduction: 621 Discussion: 1499

Conflict of Interest

The authors declare no competing financial interests

Acknowledgments

We thank Drs. Erick Pasaye, Leopoldo González-Santos, Juan Ortiz, and the personnel at the National Laboratory for Magnetic Resonance Imaging (LANIREM UNAM), for their valuable assistance. This work received support from Luis Aguilar, Alejandro De León, Carlos Flores, and Jair García of the Laboratorio Nacional de Visualización Científica Avanzada (LAVIS). Additional thanks to Azalea Reyes-Aguilar for the valuable feedback for the entire study process.

Significance Statement

In the present study, we tested whether previous observation of a facial expression associated with an emotion can modify the interpretation of a speech act that follows a facial expression; thus providing empirical evidence that facial expression impacts communication. Our results could help to understand how we communicate and the aspects of communication that are necessary to comprehend pragmatic forms of language as speech acts. We applied both multivariate and univariate analysis models to compare brain structures involved in speech acts comprehension, and found that the pattern of hemodynamic response of the frontal gyrus and medial prefrontal cortex can be used to decode classification decisions. We highlight the importance of facial expression as a relevant contextual clue for pragmatic language. Abstract

Speakers use a variety of contextual information, such as facial emotional expressions for the successful transmission of their message. Listeners must decipher the meaning by understanding the intention behind it (Recanati, 1986). A traditional approach to the study of communicative intention has been through speech acts (Escandell, 2006). The objective of the present study is to further the understanding of the influence of facial expression to the recognition of communicative intention. The study sought to: verify the reliability of facial expressions recognition, find if there is an association between a facial expression and a category of speech acts, test if words contain an intentional load independent of the facial expression presented, and test whether facial expressions can modify an utterance’s communicative intention and the neural correlates associated using univariate and multivariate approaches. We found that previous observation of facial expressions associated with emotions can modify the interpretation of an assertive utterance that followed the facial expression. The hemodynamic brain response to an assertive utterance was moderated by the preceding facial expression and that classification based on the emotions expressed by the facial expression could be decoded by fluctuations in the brain’s hemodynamic response during the presentation of the assertive utterance. Neuroimaging data showed activation of regions involved in language, intentionality and face recognition during the utterance’s reading. Our results indicate that facial expression is a relevant contextual cue that decodes the intention of an utterance, and during decoding it engages different brain regions in agreement with the emotion expressed. key words: communicative intention, speech acts, face expression, pragmatic language, functional magnetic resonance imaging.

2 Introduction

Language-communication has been described as the process by which distinct sub-processes converge and mental representations of phonological stimuli lead to recognition of meanings, which depend on our understanding of the words themselves and of the speaker’s intention. Assumptions and expectations are provided partially by the contextual frame, and are used to interpret the utterance at the lexical, grammatical and pragmatic levels (Friederici, 1999; Yule, 2010). For this reason, it is important to consider the circumstances in which the words are produced, that is, who uses them, when, and with what intention (Reyes, 1995).

Intention recognition, defined as the motive for an action in order to produce an outcome, is one of the main linguistic processes necessary to decode language and its pragmatic forms such as speech acts (Holtgraves, 1986; Catmur, 2015). These pragmatic forms require that the speaker use contextual cues, including shared knowledge, gestures, tone and facial expressions. The receiver must decode the message by retrieving the literal (semantic) meaning, and using inferential processes that take into account the beliefs, attitudes, emotions and mental state of the speaker, also known as theory of mind (Van Dijk, 1977; Frith and Frith, 2006; Reyes-Aguilar et al., 2018).

Among the contextual cues, facial expression has been described as an one of the most important stimuli that helps detect communicative intention due to the ease of emotion extraction, a skill that begins to develop early on, through socialization (Calder and Young, 2005; Cleveland et al., 2006; Hehman et al., 2015). Previously, Domaneschi et al., (2017) had reported an association between action-units (AU) on the upper face, and utterance defined as speech acts. In particular, there appeared to be specific combinations of AU on the upper face that can provide non-verbal cues and contribute to the interpretation of speech acts (SA) in certain instances. Briefly, SA may be defined as communicative acts through which speakers achieve something in a specific context, such as promise, thank, and order. They can be divided into three acts, of which the illocutionary act involves the intention of the speaker (Oishi, 2006; Lee, 2016; Domaneschi et al., 2017; Licea-Haquet et al., 2019).

Neurocognitive mechanisms underlying communicative intention in language engage several brain regions. Prior imaging studies have described the participation of the core language network, theory of mind, mirror neuron system and a common neural network related to communicative intent (Reyes-Aguilar et al., 2018). The intentional network that includes the

3 precuneus (Pcu), superior temporal sulcus (STS), parieto-temporal junction (TPj) and inferior frontal gyrus (IFG), supports the decoding of dissimilar meanings even from the same utterance within a different context (Enrici et al., 2011; Bosco et al., 2017; Tettamanti et al., 2017; Schütz et al., 2020).

The purposes of this study were, first, to find out if a sensory input, such as facial expression, can modify the interpretation of an assertive utterance, and thus its communicative intention. Second if the hemodynamic brain response (BOLD) to an assertive utterance, was similarly modulated by the previous exposure to a particular facial expression. To achieve this, we used a series of classification tasks to test if there was a consistent relation between categories of SA and facial expressions. First, we verified that participants could reliably and consistently recognize facial expressions, and we tested if recognition was affected by the sex or ethnicity of the facial stimuli. Second, we evaluated if participants showed a priori an association of a facial expression with a category of SA. We also tested if words per se contained an intentional load independent of the facial expression presented. Third, we evaluated whether facial expressions could modify an utterance’s communicative intention. Finally, we evaluated the hemodynamic brain response to the assertive utterances after previous exposure to different facial expressions.

Experimental procedures

Experiment 1 (Facial expression recognition)

To evaluate the recognition of facial expressions in an adult Mexican population, we used three databases (see below for details) and evaluated the stimuli using an online survey (Google Forms, SFigure 1, supplementary material). We selected the database with the highest scores for subsequent experiments.

Participants. The survey was answered by 52 Mexican participants with Spanish as their native language (mean age 23.67 ± 3.28; range: 18 - 63 years old; 21 women).

Selection of facial expressions databases. After a wide online search, we chose three databases that included facial expressions that represented each of the six basic emotions (according to Ekman, 1970) and a neutral face, with actors of Caucasian and/or Latin American ethnicity, both males and females. These were the Montreal Set of Facial Displays of Emotion or MSDF (Beaupré and Hess, 2005), Set of Emotional Facial Expression Pictures

4 or WSEFEP (Olszanowski et al., 2015), and Compound Facial Expressions of Emotion or CFEE (Du et al., 2014).

Procedure. From each database we selected twelve actors, male and female, of Latin American ethnicity. Participants were asked to select which facial expression (joy, anger, sadness, disgust, surprise, fear and neutral) corresponded to one of three emotions. The question asked was: Which of these images expresses anger/joy/neutrality? Participants were also asked if they considered that the actor belonged in their community (Could this person be a member of your community?). Each participant rated a total of 84 facial expressions.

Statistical analysis. The statistics were done using R3.5.0 (R Core Team, 2020). We compared the frequencies of correct categorization of the three facial expressions considering all databases. A Levene test was performed, which showed that the variances were not homogeneous (21.65, p <0.001), so we used the Friedman test and a post hoc test with Bonferroni correction. The Wilcoxon test for related samples was used to compare the frequencies of correct categorization of facial expressions between ethnicities.

Results. The results showed that the facial expressions most accurately categorized (i.e., greater mean frequency of correct responses) by the 52 participants were joy (50.42 ± 1.25), followed by the neutral expression (46.5 ± 0.71) and anger (30.96 ± 8.07); and that these were categorized with different degrees of precision (Fr = 53.626, gl= 2, p < 0.001). With respect to the accuracy of responses depending on the ethnicity of the actor, the results indicated that there were no significant differences in the accuracy of responses between ethnicities (Caucasian 43.1852 ± 8.52; Latin American 42.36 ± 10.53). Regarding differences between male and female actors, participants accurately categorized male (42.52 ± 9.41) and female (42.73 ± 10.45) facial expressions. With respect to the question of whether the faces could belong to their community, we found that participants responded that the actors in the CFEE database were recognized as potential community members to a greater degree (82.56 ± 6.23) than MSDF (74.99 ± 8.02) and WSEFEP (72 ± 2.82) databases.

5 Figure 1. Percentage (mean±standard deviation) of correct categorization of facial expressions including all databases considering (A) emotional facial expressions, (B) ethnicity, and (C) sex of the actor in the database.

Experiment 2 (Association between facial expressions and speech acts)

Based on the results of Experiment 1 we selected the CFEE database for all subsequent experiments. This experiment allowed us to establish if certain facial expressions were associated a priori with specific speech acts, that is, if the facial expressions themselves indicated an intention.

Participants. The experiment was conducted with a sample of 40 Mexican participants (24.42 ± 5.99, 20 women, range: 18 - 56 years old).

Experimental design. Using an online survey (Google Forms, SFigure 2) we asked if there was an association between speech acts (four different categories were chosen) and the six facial expressions, plus neutral, selected from the CFEE database. Six actors, with all six facial expressions plus neutral, were selected based on the accuracy scores obtained in Experiment 1. Each facial expression was presented along with the following question: Which of the following image(s) expresses an order/petition/demand/earnest request? Each participant selected the emotional expression(s) that best fit what they considered to be the speech act in question.

6 Statistical analysis. A Pearson's Chi-squared test for analysis of independence was carried out to evaluate the association between the categorical variables, facial expression and speech act, according to the frequency of responses. A mosaic plot based on Pearson standardized residuals was used to visualize the contribution of each variable to the statistical result (R, library corrplot, RRID:SCR_018081).

Results. The results indicated a significant association between the facial expressions and speech acts [X²(18, 260) = 615.47; p < 0.001] (Figure 2). The preferred combinations in terms of selection frequency were: order - neutral face (mean = 22.16), order - anger (mean = 11.33), earnest request - sadness (mean = 18.16), earnest request - fear (mean = 13), demand - anger (mean = 16), demand - neutral (mean = 11.8).

Figure 2. Mosaic plot showing the strength and direction of the association between categories of speech acts and facial expressions based on Pearson standardized residuals. Warm colors indicate the compatibility (positive values), and cold colors indicate the incompatibility (negative values) between speech acts and facial expressions.

Experiment 3 (The effect of facial expressions on the categorization of intention)

For this experiment, different facial expressions were presented before an assertive utterance, and the participants were asked to categorize the utterance as an assertion, a demand or an earnest request. The facial expressions and linguistic stimuli were selected based on the results of the previous experiments.

7 Participants. Two samples of participants were selected for the evaluation of the stimuli. Each sample was composed of 20 right-handed participants (10 women), Mexican nationals and Spanish as their native language. The first sample, with an average age of 25.7 years (± 3.32), answered half of the total statements. The second sample with an average age of 25.6 (± 3.28) answered the other half of the statements. Each of the participants signed an informed consent form.

Experimental design. We selected four actors from the CFEE, whose facial expressions were most accurately categorized in the previous experiments. We used an Asus SL301 computer for the presentation of the stimuli and the recording of the behavioral responses with PsychoPy v. 1.85.6, RRID:SCR_006571 (Peirce, 2007). An optometrist’s positioner was used to keep the gaze fixed on and directed towards the presented face, a response device was used for the generation of responses (Five Button Response Pad, OTR-1X5-N4 from Current Designs). A total of 30 linguistic stimuli structured as assertive utterances were designed, verbs with neutral valence were selected and verified with the Sentiment and Emotion Lexicons (Mohammad and Turney, 2010). The statements had a mean word length of 5.3 (± 1.24, range = 7 - 4). A group of 10 judges verified that the linguistic stimuli were interpreted as statements, i.e., an assertive utterance, that states what “the speaker believes to be the case or not” (Yule, 1996, pp 53). The 30 linguistic stimuli generated were divided into two parts for the behavioral assessment by 20 participants. For each sample of 15 statements the frequency of use of fraction per million was verified with Sketch Engine (Kilgarriff et al., 2014)(http://www.sketchengine.eu), we expected to find a normal distribution for all verbs. Each stimulus was presented on the screen, the participant’s gaze was fixed by the positioner. We used the sequence described by (Holtgraves, 2008) to evaluate the priming effect. Briefly, a fixation cross preceded each block, then one of three facial expressions was presented, for a variable duration of 250 or 2500 ms, followed by a fixation cross of 100 ms. The statement (assertive utterance) was then presented, the duration of which depended on the participant's response, then the participant was asked to categorize the statement as a demand, assertion or earnest request using the response device. Participants were trained on this task and were asked to read the statement as an utterance produced by the person whose face he or she had observed. At the end of the task, the participants were asked to classify each verb read according to the speech act categories.

8 Statistical analysis. The “Power Analysis with Crossed Random Effects" software (https://jakewestfall.shinyapps.io/crossedpower/) developed by Westfall and collaborators (2014) was used to calculate the number of stimuli needed to obtain adequate statistical power, assuming a variation among participants, conditions, stimuli and interactions as reported in other research, as well as the assumption of an effect size of 0.5 (Westfall et al., 2014). From the data obtained, a Pearson's Chi-squared test was calculated in the same way as in the previous experiment for the verification of compatibility between categories. Using the Shapiro-Wilk statistic, the distribution of the frequency of use of verbs for each sample was evaluated. For the first 15 verbs the distribution resembled the normal distribution (W = 0.97, p > 0.91), the same occurred for the remaining 15 verbs (W = 0.93, p > 0.3). Cochran's Q test for related samples and dichotomous variables, and a post-hoc analysis were used to determine the differences in responses for each verb.

Results. The chi-square analysis showed that participants associated specific facial expressions with speech acts in the following way: joy-assertion, anger-demand and sadness- earnest request [X²(4, 4560) = 11877; p < 0.001]. The time of presentation of facial expressions (250 or 2500 ms) did not have an effect. Regarding the classification of verbs as belonging to one of the speech act categories used in the present study, it was found that almost all verbs were classified as assertions, with significant differences (p < 0.05), compared to the other categories, i.e., demand and earnest request (repeated measures ANOVA for each verb; Figure 4). The verb “to replace” (sustituir) was classified more frequently as a demand, however, no significant difference was found when compared to the category assertion; the same occurred with the verbs “to review” (revisar) and “to complete” (finalizar).

9 Figure 3. Mosaic plot showing the strength and direction of the association between categories of speech acts and facial expressions based on Pearson standardized residuals. A significant association (p < 0.001) is observed between category of speech acts (assertion, demand and earning request) and facial expressions (joy, anger and sadness). Warm colors indicate compatibility (positive values), and cold colors indicate incompatibility (negative values) between speech acts and facial expressions.

10 Figure 4. Classification of verbs in the different categories of speech acts (percentages). Significance set at p < 0.05; according to repeated measures ANOVA.

Experiment 4 (fMRI experiment)

Participants. Twenty-two participants (age 21.68 ± 2.81, 11 women) with Spanish as their native language, took part in the fMRI experiment. All participants were healthy, with 12 years or more of formal education, had normal hearing, normal or corrected to normal vision, normal verbal comprehension measured by the Wechsler Adult Intelligence Scale (mean: 108 ± 13.33; (Wechsler, 2012), and right-handed laterality according to the Edinburgh handedness inventory (mean: 0.69 ± 0.39; (Oldfield, 1971). No neurological or psychiatric

11 disorders were reported during a screening interview and psychological distress was within the norm, according to the Mexican version of the Symptom Checklist-90 (González-Santos et al., 2007). Before the experiment, participants were informed about the study purposes and procedures, after which they provided their written informed consent. All procedures complied with the local ethics committee on the Use of Humans as Experimental Subjects approved by the internal Committee on Ethics (0.71H-RM), which also approved the experimental protocol, in compliance with the federal guidelines of the Mexican Health Department (http://www.salud.gob.mx/unidades/cdi/nom/compi/rlgsmis.html), which agree with international regulations.

Neuropsychological tests. To measure the cognitive status of each participant, a battery of psychometric tests were applied to the participants. For the assessment of executive functions, the N-back, Digit Span, Tower of London and Go-no-go tests, contained in the The Psychology Experiment Building Language software, RRID:SCR_014794 (Mueller and Piper, 2014), and the local-global test based on the work of Miyake (2000) using Psychtoolbox-3, RRID:SCR_002881 (Brainard, 1997) were used. In addition, the Wechsler Adult Intelligence Scale (Wechsler, 2012) was applied for the evaluation of perceptual and verbal coefficient, Interpersonal Reactivity Index for Empathy (Lucas-Molina et al., 2017) and for Theory of Mind: the Short Story task (Dodell-Feder et al., 2013) and Reading Mind in the Eyes Test (Baron‐ Cohen et al., 2001). Full applied test and their results are presented in Table 1.

Experimental design. We used the stimuli that were validated in Experiments 1 and 3. Briefly, thirty statements and three facial expressions (anger, sadness, and happiness) with the six actors that were better recognized and initially selected from the CFEE database. As a control for the fMRI task, a blurred face was created using the graphics editor GIMP v. 2.10.2, taking as a model the emotional neutral face from each actor. The result was a 30 x 4 design of four experimental conditions depending on the facial expressions (anger, sadness, happiness) and blurred face, for a total of 120 stimuli (Figure 5), so that all statements followed all facial expressions and the blurred face. Events in functional magnetic resonance scanning sessions were presented semi-randomly in an event-related paradigm following the tool for automatically scheduling events for rapid-presentation event-related (OptSeq, RRID:SCR_014363: http,//surfer.nmr.mgh.harvard.edu/optseq) and DesignDiagnostic (https://montilab. psych.ucla.edu/designdiagnostics/) for verification of multicollinearity and

12 efficiency in order to achieve a better BOLD signal for each event. Stimuli were presented on a black background via PsychoPy software (Peirce, 2007). NordicNeuroLab goggles (Bergen, Norway) were used for task presentation and a response device (Cedrus Lumina, California, USA), both compatible with the scanner.

Figure 5. Example of the experimental design for the fMRI Speech Act- Classification task, translated into English. Statement example: Natalia watches the movie.

Procedure. For the fMRI paradigm we used two different tasks. The first one (Speech Act- Classification task) consisted of a facial expression or blurred face followed by an assertive utterance. The facial expression or blurred face were presented during 500 ms, since previously we found no differences between 250 and 2500 ms, a time of 500 ms was chosen to allow the subjects to obtain more information from the face. Subsequently, a fixation cross was presented for 100 ms, followed by an assertive utterance for 2500 ms. Finally, participants were asked to classify the statement read as an assertion, demand, or earnest request, answers were obtained using a response device. Each possible answer was randomly located on each screen corner (Figure 5). Participants were instructed to answer in the blank screen corner if none of the possible answers satisfied the statement interpretation. In addition, a fixation cross was added prior to each event, to achieve a jittering effect, i.e. a

13 variable and unpredictable interval, with a duration determined by OptSeq2. The second task (Facial Expression Classification task) was identical to the first task, with one exception, instead of the statement only a fixation cross was presented. For this task the participants were asked to classify the facial expression in terms of the emotion portrayed. Before the start of the scanning session, the participants responded to five test trials to verify that they understood the tasks. Then, they entered the scanner, an initial resting state scan was performed for 10 minutes with eyes closed, then the Speech-Act Classification task in three runs, followed by the Facial Expression Classification task. At the end of the runs, a high resolution T1 scan was obtained.

Neuroimaging data acquisition. fMRI imaging acquisition was performed on a 3.0T GE MR750 scanner (General Electric, Waukesha, WI, USA) using a 32-channel head coil. Functional images were acquired using a T2*-weighted EPI sequence included in 38 slices (TR= 2000 ms, TE= 40 ms, a 64 x 64 matrix and final voxel size= 4 x 4 x 4 mm3 isometric volume, FOV= 25.6 cm). High-resolution structural images were acquired using a 3D T1- weighted sequence of whole-brain (resolution= 1 x 1 x 1 mm3, TR= 2.3 s, TE= 3 ms) for anatomical localization.

Neuroimaging univariate analysis. MRI data preprocessing was performed with the fMRIprep pipeline v.1.1, RRID:SCR_016216 (Esteban et al., 2019), including correction for slice-timing, head motion, and normalization onto MNI common brain space (EPI template, 2 x 2 x 2 mm). Statistical analyses were made using FMRI Expert Analysis Tool using FMRIB’s Improved Linear Model (FEAT FILM) v.6.0 from FSL, RRID:SCR_002823 (FMRIB’s software library, www.fmrib.ox.ac.uk/fsl). Blood oxygen level-dependent (BOLD) signal was analyzed during utterance presentation. Statistical analysis of event-related hemodynamic changes was carried out as per the general linear model (GLM, Friston et al., 1995). The first-level fMRI analysis was performed for each of the four conditions with a significance threshold of Z > 2.3. The mid-level analysis was carried out using a fixed-effects model to determine brain regions involved in the activity of each condition and the contrast between them. To identify activation at the group level, a FLAME 1 (FMRIB’s Local Analysis of Mixed Effects) for third-level analysis was used with a cluster-significance threshold criterion of Z > 2.3 with p < 0.05, corrected with Gaussian Random Field (GRF) for whole-brain level results. The anatomical regions were determined with the Harvard-Oxford atlas of cortical and subcortical regions (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Atlases) with the

14 coordinates corresponding to MNI152 template. The regressors included Anger, Sadness, Joy, and Blurred for both tasks. The average activity of each regressor was also contrasted between them as follows: Joy > Blurred, Anger > Blurred, Sadness > Blurred, Blurred > Joy ∪ Anger ∪ Sadness (JAS), JAS > Blurred, as well as a contrast between each pair of emotions. The Speech-Act Classification task was contrasted with the Facial Expression Classification task in order to discard the possible carryover effect of facial expression observation.

Neuroimaging multivariate analysis. To determine whether any region was implicated in decoding the classification decision we performed a multivariate analysis approach using a searchlight-based multivariate support vector implemented in the Decoding Toolbox, RRID:SCR_017424 (Hebart et al., 2014) within SPM12 r7758, RRID:SCR_007037 in MATLAB R2018b (MathWorks Inc., Natick, MA USA). The support vector machine with cost parameter (C=1) was trained to classify brain activity patterns from the four conditions (Joy, Anger, Sadness and Blurred) during the Speech-Act Classification task in order to identify brain regions that encode the classification decision. Then we ran the searchlight- based analysis for each subject, obtaining a whole-brain accuracy-minus-chance map averaging the accuracy across all runs. For each time bin, we used a spherical cluster of 5 voxel-radius, and the center was moved throughout the brain. Finally, our searchlight maps were normalized to MNI space and smoothed and averaged across all subjects with a 6 mm full-width half-maximum (FWHM) kernel, and corrected using FDR.

Psycho-Physiological Interaction (PPI): The above-chance classification performance on MVPA was higher on the middle frontal gyrus (MFG), a region also found on univariate analysis contrasts (STable 5). Therefore, we evaluated if the MFG interacted with other regions during utterance reading, using a PPI approach. We created a mask in the non-border highest peak obtained from MVPA (52 22 38 MNI coordinates) with an inflated 4 mm radius.

Statistical Analysis. Behavioral data from both classification tasks were analyzed using chi- square analysis as in the previous experiments. To evaluate whether there are psychometric variables that could predict BOLD signal change, a forward-backward stepwise multiple linear regression analysis was performed, using the minimum Akaike Information Criterion (AIC) to select the best model after removing the multicollinearity. For this purpose, the raw scores of the psychometric tests applied were obtained, and two multiple regression analyses were calculated. The first for executive functions and the second for theory of mind (Table

15 1). For both analyses the dependent variable was the average change of the BOLD signal in each of the maximum peaks obtained in the contrast of each emotion rather than the blurred face in univariate analysis: Paracingulate gyrus (Joy), Angular gyrus (Anger), Lingual gyrus (Sadness), Fusiform gyrus (Blurred). For each stepwise regressions, the statistical requirements were checked for compliance. All the independent variables were scalar data, and were analyzed using the Shapiro-Wilk test to check whether the residual error for each of the models generated was normally distributed. Except for one, the models were normally distributed. The Breusch-Pagan test was used to determine the existence of heteroscedasticity. This test indicated that there was no evidence of lack of homoscedasticity. A variance inflation analysis was also performed, which showed that none of the predictors maintained a very high linear correlation (multicollinearity) or variance inflation. Nor was autocorrelation between variables detected by the Durbin-Watson test.

Results

Behavioral results. The results for the Speech-Act Classification task showed similar results to previous experiments [X²(9, 16) = 1047.8, p < 0.001], with a predominance in the association between Joy-Assertion, Anger-Demand and Sadness-Earnest request with a weaker association in general (Figure 6A). While for the Facial Expression Classification task, associations between facial expressions and emotions were as expected, participants classified the facial expression as they were classified in the CFEE database [X²(9.16) = 638.37, p < 0.001], with the blurred-face condition showing no significant associations. Reaction times for each combination were significantly different [W (15, 16) = 73.931, p < 0.001]. Anger-Demand combination required significantly more time for classification (Figure 6B).

16 Figure 6. A) Mosaic plot showing the strength and direction of the association between categories of speech acts (SA) and facial expressions based on Pearson standardized residuals. Warm colors indicate compatibility (positive values), and cold colors indicate incompatibility (negative values) between SA and facial expressions. B) Violin plots showing median reaction time for the classification task for each combination of SAt and facial expression. The box plot represents the interquartile range, the whiskers the rest of the distribution, and the colored area the density curve. There was a significant difference between combinations [W (15, 16) = 73.931]. *** p<0.001 post hoc Kruskal-Wallis.

Neuroimaging univariate results. For the Speech-Act Classification task, to identify the neural correlates of speech acts comprehension after observation of a facial expression we examined the contrasts for each Facial expression > Blurred. Then we removed the BOLD response due to the observation of the facial expression by calculating the contrasts with the results of the Emotion Classification task. The contrasts were (Joy-assertive utterance > Blurred-assertive utterance) > (Joy-Face > Blurred-Face), (Anger-assertive utterance > Blurred-assertive utterance) > (Anger-Face > Blurred-Face), (Sadness-assertive utterance > Blurred-assertive utterance) > (Sadness-Face > Blurred-Face) and (Blurred > Joy+Anger+Sadness-Utterance) > (Blurred > JAS-Face).

Reading the assertive utterance after seeing a facial expression of joy, revealed activations in the Paracingulate Gyrus (Pcg) and Superior Frontal Gyrus (SFG; Figure 7A and STable 1). While anger revealed activations in the same regions, plus the Angular Gyrus, also labeled as Temporal-Parietal junction (Schurz et al., 2017), Precuneus and Frontal areas (Figure 7B and STable 2). Sadness, in addition to the areas activated by anger, showed large cluster

17 activations in Posterior Cingulate (PCC; Figure 7C and STable 3). Blurred face, on the other hand, when contrasted with all emotions, showed activations in Temporal Fusiform Cortex, Precentral, Paracingulate, Middle Temporal and other frontal areas (Figure 7D and STable 4).

Figure 7. Results of the whole brain analysis, Speech-Act Classification task contrasted with Emotion Classification task. Results of the contrasts are shown on the lateral and medial views of right and left hemispheres. A) (Joy-assertive utterance > Blurred-assertive utterance) > (Joy-Face > Blurred-Face). B) (Anger-assertive utterance > Blurred-assertive utterance) > (Anger-Face > Blurred-Face). C) (Sadness-assertive utterance > Blurred-assertive utterance) > (Sadness-Face > Blurred-Face). D) (Blurred > Joy ∪ Anger ∪ Sadness (JAS)-Utterance) > (Blurred > JAS-Face). Abbreviations: mFG = medial Frontal Gyrus, TPj/Ang = Temporal-Parietal junction / Angular Gyrus, Pcu = Precuneus, PCC = posterior Cingulate, FFG = Fusiform Gyrus.

18 Neuroimaging multivariate analysis results. Based on the Speech-Act Classification task, the Searchlight-based analysis revealed that classification decisions (Joy vs Anger vs Sadness vs Blurred) could be decoded based on three clusters located on the Frontal Gyrus (superior, middle and inferior), with an additional small cluster that extended into the Paracingulate gyrus (Pcg), within the medial Prefrontal cortex (mPFC; Figure 8A and STable 5). However, the maximum peak activation was located over the Middle Frontal Gyrus (MFG).

Psycho-Physiological Interaction results: The peak activation map found with the MVPA was used to create a mask in the MFG, and used as a seed region (Figure 8B). We found that this region showed significant functional interaction with the PCC, extending into Pcu during utterance reading after a facial expression of joy, or anger and also after a blurred face. In contrast, no such interaction was found after a facial expression of sadness (Figure 8C).

Figure 8. Results of the multivariate pattern analysis (MVPA), and psychophysiological interaction (PPI). A) MVPA results. The Searchlight-based analysis revealed that classification decisions (Joy vs Anger vs Sadness vs Blurred) during Speech act- Classification task, could be decoded based on three clusters located on the Frontal Gyrus

19 (superior, middle and inferior), with an additional small cluster that extended into the Paracingulate gyrus, within the medial Prefrontal cortex (STable 5). The maximum peak activation was located over the Middle Frontal Gyrus (MFG). B) Mask in the MFG [52 22 38 MNI coordinates] used for the PPI analysis. C) PPI results for Joy (Red), Anger (Blue), Sadness (Green) and Blurred face (Purple). Significant functional interaction activation was found in the Posterior Cingulate cortex, extending into Precuneus during utterance reading after a facial expression of joy, anger or after a blurred face. In contrast, no such interaction was found after a facial expression of sadness, see STable 6 for peak activation coordinates.

Multiple Regression Analysis. The results showed executive function tests predicted change in BOLD signal in two brain regions: paracingulate gyrus (R2 adjusted = 0.452, p = 0.005) and fusiform gyrus (FFG; R2 adjusted = 0.585, p < 0.001). The second analysis, which included tests associated with theory of mind and empathy, showed a significant predictor only within the fusiform gyrus (R2 adjusted = 0.410, p = 0.005). See supplementary table 8 and 9 for a summary of all results from this analysis.

Discussion

The purposes of this study were, first, to find out if facial expression can modify the interpretation of an assertive utterance, and thus its communicative intention. Second if BOLD response to an assertive utterance, was similarly modulated by previous exposure to a particular facial expression. Our results showed that previous observation of a facial expression associated with an emotion can indeed modify the interpretation of an assertive utterance that followed the facial expression. Results also showed that BOLD response to an assertive utterance was modulated by the preceding facial expression. Furthermore, we found that classification decisions based on the emotions expressed by the face could be decoded by fluctuations in BOLD response during the presentation of the assertive utterance that did not include any emotional or intentional content. The relevant regions included the Frontal gyrus, mPFC including the Pcg. The mFG showed significant functional interaction with the PCC, extending into Pcu during utterance reading after a facial expression of joy, anger and also after a blurred face. Together these results indicate that facial expression is a relevant contextual cue used to decode the intention of an utterance, and during decoding it engages different brain regions in consonance with the emotion expressed.

20 Previous studies had shown that specific combinations of AU on the upper face can provide non-verbal cues and contribute to the interpretation of SA (Domaneschi et al., 2017). In this study we used whole-face expressions of basic emotions (Du, et al., 2014) and found an association between the expression of anger and a demand. The relationship found between anger and demand could indicate that extra- and paralinguistic elements accompany specific everyday phrases. In the present case, results suggest that when demanding, the facial gesticulation pattern is similar to anger, while an earnest request is associated with expressions of sadness and fear. Additionally, behavioral results of experiment 4 showed that the emotional expression of the face immediately preceding an utterance structured as an assertion, modified the categorization of that utterance. Hence, faces expressing anger resulted in assertive utterances being categorized as demands, sadness faces resulted in being categorized as earnest requests, and joy faces resulted in as assertions. It is important to emphasize that the statements did not change, they were the same statements. Therefore, it can be inferred that participants can increase or decrease the illocutionary force of an utterance even when dealing with the same propositional content, depending on the facial expression that accompanies the utterance. It is worth noting that the facial expression of sadness was related to an earnest request to a much less degree, which could be because, as a linguistic expression, an earnest request is related to a distinct configuration of action-units that may include sadness. Similar to the results of Domaneschi et al. (2017), where extralinguistic gestures of particular speech acts (SA) could be combinations of AU. Behavioral results of all the experiments indicate that faces work as vehicles for the expression of the illocutionary force of utterances, in the same way as manual and head gestures are used to disambiguate, enhance or highlight information (Wagner et al., 2014). The main functions of gestures are to contribute to the utterance content, and help with aspects related to the particular situation (Kendon, 2004 cited in Wagner et al., 2014). In other words, the comprehension of an utterance depends on the context in which it is produced, in this case, the presence of an emotionally expressive face. Comprehension of an utterance in a certain context entails the decoding of the literal or propositional meaning and inferential processes related to the speaker's intentionality and also the listener’s beliefs, attitudes and emotions (Panther and Thornburg, 1998; Hagoort, 2017). The present results show that there is a difference in the inferential processing of an utterance’s intention when reading it after observing a specific facial expression, and when reading it without the contextual information that a facial expression may bring.

21 Univariate and multivariate results from fMRI-experiment showed comprehension of assertive utterances after seeing an emotional face expression implies the activation of brain networks involved in processes such as communicative intention, theory of mind and language (Enrici et al., 2011; Fedorenko and Thompson-Schill, 2014; Schurz et al., 2014). In univariate analysis, the incremental cluster activation from facial expressions of joy, to anger and to sadness could imply a higher demand for brain resources depending on the emotion expressed. This result may indicate a more complex processing for each emotion in relation to its role in a social context. Emotional expressions may function as a communicative signal of differentiation and conflict with the external environment, in interpersonal relationships and affective bonds, and could have a greater interpretative weight for making a decision, recruiting networks of social cognition (Reyes-Aguilar et al., 2018). Notwithstanding, it is important to emphasize the lower rate of association between a facial expression of sadness and the interpretation of an assertion as an earnest request. The greater activation may represent higher cognitive demand because the facial expression may not be giving enough information to drive the inference. As mentioned before, the study by Domaneschi et al. (2017), suggests that extralinguistic gestures of particular SA could be combinations of AU. On the other hand, the blurred face did not convey information about the facial expression (emotion), however, we found that it engaged a variety of brain areas, and that there was a significant functional interaction between the MFG and other brain areas, including FFG, a common region activated by faces (Kanwisher et al., 1997). This could indicate that even if there is no information about emotion, the blurred face conveys information, clearly of a different kind, possibly the absence of information about the facial expression of the speaker and the cognitive effort of trying to decode the facial features. In our searchlight-based analysis we found that MFG jointly with mPFC play a role in classification decisions (e.g., joy vs anger). Kotz et al., (2013) found that when presented with vocal expressions portraying different emotions, a right-lateralized network that included the IFG, anterior STS, and MFG were implicated in discriminating between emotions when using the same searchlight-based analysis. The mPFC and MFG play an important role in decoding emotional states, multiple studies have indicated that activity within the mPFC is involved in the discrimination of emotional states (Lane et al., 1997; Reiman et al., 1997; Spotorno et al., 2012). However, there is little evidence for the distinction between the type of emotions that evoke specific activation patterns (Phan et al., 2002). Additionally, other studies have emphasized the role of MFG and mPFC in the linguistic understanding of SA and pragmatic abilities (Rapp et al., 2012; Reyes-Aguilar et

22 al., 2018). In this sense, (Bosco et al., 2017) described the involvement of MFG and PFC in the recognition of ironic and deceitful communicative intentions. In short, the role of MFC and mPFC in intentional communication could be important for decision making, along with their connectivity with other regions involved in various cognitive processes, such as the Pcu and the PCC, as found here in the PPI analysis. The results of the multiple linear regression showed that it is possible to roughly predict the behavior of the BOLD activity signal change in Pcg and FFG, based on the psychometric test scores in regions previously associated with communicative intention and with executive tasks such as Digit-Span and Tower of London (Enrici et al., 2011; Piper et al., 2015). Predicted BOLD activity could highlight the importance of a range of cognitive resources implicated on communicative intention decoding. Similarly as Bosco et al., (2018), who found that when including ToM and EF scores in analysis, they could explain a moderate amount of the variance in pragmatic performance, however their study was focused on patients with traumatic brain injury, and their results might not be generalizable to a healthy population. The fMRI Speech-Act Classification task performed in this study involves processes of attentional exchange, updating, planning, and decision making, so a variation in the signal of the regions commonly associated with these processes was to be expected as we observed in the multiple regression analysis. However, results described should be interpreted with caution as there is still no consensus on the relationship between complex cognitive processes for task resolution with the activation of the brain regions found here. The main limitations of this paper are the number of SA and emotional expressions utilized. The ones we selected were those that showed reliable recognition and association between them. Our conclusions are therefore restricted to this set, and further studies are needed to evaluate if this is the case for more complex facial expressions, and other kinds of SA. Another issue that remains unanswered is whether these results would be the same in a sample of a population from a different culture. Although we found reliable recognition of emotions from facial expressions in our participants, there is data indicating that this may not always be the case (e.g. Feldman Barrett et al., 2019).

Conclusion

23 In conclusion, this study agrees with previous evidence showing that facial expressions play an essential part in conveying the communicative intention of speech acts. The results from our fMRI study, reinforce the idea that the comprehension of speech acts is supported by brain networks that are also involved in processes such as theory of mind and executive functions. Furthermore, specific emotions might involve more complex processing by the individual than others. Further research is needed to understand the ecological validity of our findings.

Data sharing

The fMRI dataset for this study can be found in https://openneuro.org/datasets/ds003481/ described as “Speech acts - Emotion categorization task”, from subject 025 to 046 (Rasgado- Toledo et al., 2021).

Code Accessibility

Code and workflow are available in: https://jalilrt.github.io/Facial-expresion-in- communicative-intention/

Funding

This study was supported by grants from UNAM, DGAPA-PAPIIT IN203818-2, and the Mexican Council for Science and Technology, Fronteras de la Ciencia CONACyT 2015 no. 225. to M. Giordano, and scholarship #476261 to R-T, J., and 755580 to E. V-C from intramural funds (Instituto de Neurobiología, UNAM).

24 References

Baron‐ Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I (2001) The “Reading the Mind in the Eyes” Test Revised Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High‐ functioning Autism. Journal of Child Psychology and Psychiatry 42:241–251 Available at: http://dx.doi.org/10.1111/1469-7610.00715.

Barrett LF, Adolphs R, Marsella S, Martinez AM, Pollak SD (2019) Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest 20:1–68 Available at: http://dx.doi.org/10.1177/1529100619832930.

Beaupré MG, Hess U (2005) Cross-Cultural Emotion Recognition among Canadian Ethnic Groups. Journal of Cross-Cultural Psychology 36:355–370 Available at: http://dx.doi.org/10.1177/0022022104273656.

Bosco FM, Gabbatore I, Angeleri R, Zettin M, Parola A (2018) Do executive function and theory of mind predict pragmatic abilities following traumatic brain injury? An analysis of sincere, deceitful and ironic communicative acts. J Commun Disord 75:102–117 Available at: http://dx.doi.org/10.1016/j.jcomdis.2018.05.002.

Bosco FM, Parola A, Valentini MC, Morese R (2017) Neural correlates underlying the comprehension of deceitful and ironic communicative intentions. Cortex 94:73–86 Available at: http://dx.doi.org/10.1016/j.cortex.2017.06.010.

Brainard DH (1997) The Psychophysics Toolbox. Spatial Vision 10:433–436 Available at: http://dx.doi.org/10.1163/156856897x00357.

Calder AJ, Young AW (2005) Understanding the recognition of facial identity and facial expression. Nature Reviews Neuroscience 6:641–651 Available at: http://dx.doi.org/10.1038/nrn1724.

Catmur C (2015) Understanding intentions from actions: Direct perception, inference, and the roles of mirror and mentalizing systems. Conscious Cogn 36:426–433 Available at: http://dx.doi.org/10.1016/j.concog.2015.03.012.

Cleveland A, Kobiella A, Striano T (2006) Intention or expression? Four-month-olds’ reactions to a sudden still-face. Infant Behav Dev 29:299–307 Available at: http://dx.doi.org/10.1016/j.infbeh.2005.12.004.

Dodell-Feder D, Lincoln SH, Coulson JP, Hooker CI (2013) Using fiction to assess mental state understanding: a new task for assessing theory of mind in adults. PLoS One 8:e81279 Available at: http://dx.doi.org/10.1371/journal.pone.0081279.

Domaneschi F, Passarelli M, Chiorri C (2017) Facial expressions and speech acts: experimental evidences on the role of the upper face as an illocutionary force indicating device in language comprehension. Cogn Process 18:285–306 Available at: http://dx.doi.org/10.1007/s10339-017-0809-6.

Du S, Tao Y, Martinez AM (2014) Compound facial expressions of emotion. Proceedings of the National Academy of Sciences 111:E1454–E1462 Available at:

25 http://www.pnas.org/cgi/doi/10.1073/pnas.1322355111.

Enrici I, Adenzato M, Cappa S, Bara BG, Tettamanti M (2011) Intention Processing in Communication: A Common Brain Network for Language and Gestures. Journal of Cognitive Neuroscience 23:2415–2431 Available at: http://dx.doi.org/10.1162/jocn.2010.21594.

Fedorenko E, Thompson-Schill SL (2014) Reworking the language network. Trends Cogn Sci 18:120–126 Available at: https://www.ncbi.nlm.nih.gov/pubmed/24440115.

Friederici AD (1999) Language Comprehension: A Biological Perspective (Friederici AD, ed). Berlin, Heidelberg: Springer. Available at: http://dx.doi.org/10.1007/978-3-642- 59967-5.

Frith CD, Frith U (2006) The Neural Basis of Mentalizing. Neuron 50:531–534 Available at: http://dx.doi.org/10.1016/j.neuron.2006.05.001.

González-Santos L, Mercadillo RE, Graff A, Barrios FA (2007) Versión computarizada para la aplicación del Listado de Síntomas 90 (SCL 90) y del Inventario de Temperamento y Carácter (ITC). Salud mental 30.

Hagoort P (2017) The core and beyond in the language-ready brain. Neurosci Biobehav Rev 81:194–204 Available at: http://dx.doi.org/10.1016/j.neubiorev.2017.01.048.

Hebart MN, Görgen K, Haynes J-D (2014) The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data. Front Neuroinform 8:88 Available at: http://dx.doi.org/10.3389/fninf.2014.00088.

Hehman E, Flake JK, Freeman JB (2015) Static and Dynamic Facial Cues Differentially Affect the Consistency of Social Evaluations. Pers Soc Psychol Bull 41:1123–1134 Available at: http://dx.doi.org/10.1177/0146167215591495.

Holtgraves T (1986) Language structure in social interaction: perceptions of direct and indirect speech acts and interactants who use them. J Pers Soc Psychol 51:305–313 Available at: http://dx.doi.org/10.1037/0022-3514.51.2.305.

Holtgraves T (2008) Automatic intention recognition in conversation processing☆ . Journal of Memory and Language 58:627–645 Available at: http://dx.doi.org/10.1016/j.jml.2007.06.001.

Kanwisher N, McDermott J, Chun MM (1997) The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci 17:4302–4311 Available at: http://dx.doi.org/10.1523/JNEUROSCI.17-11-04302.1997.

Kilgarriff A, Baisa V, Bušta J, Jakubíček M, Kovář V, Michelfeit J, Rychlý P, Suchomel V (2014) The Sketch Engine: ten years on. Lexicography 1:7–36 Available at: https://doi.org/10.1007/s40607-014-0009-9.

Kotz SA, Kalberlah C, Bahlmann J, Friederici AD, Haynes J-D (2013) Predicting vocal emotion expressions from the human brain. Hum Brain Mapp 34:1971–1981 Available at: http://dx.doi.org/10.1002/hbm.22041.

26 Lane RD, Reiman EM, Ahern GL, Schwartz GE, Davidson RJ (1997) Neuroanatomical correlates of happiness, sadness, and disgust. Am J Psychiatry 154:926–933 Available at: http://dx.doi.org/10.1176/ajp.154.7.926.

Lee EL (2016) Language and culture (Oxford Research Encyclopedia, Communication, ed). Oxford University Press. Available at: https://doi.org/10.1093/acrefore/ 9780190228613.013.26.

Licea-Haquet GL, Velásquez-Upegui EP, Holtgraves T, Giordano M (2019) Speech act recognition in Spanish speakers. J Pragmat 141:44–56 Available at: http://dx.doi.org/10.1016/j.pragma.2018.12.013.

Lucas-Molina B, Pérez-Albéniz A, Ortuño-Sierra J, Fonseca-Pedrero E (2017) Dimensional structure and measurement invariance of the Interpersonal Reactivity Index (IRI) across gender. Psicothema 29:590–595 Available at: http://dx.doi.org/10.7334/psicothema2017.19.

Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD (2000) The Unity and Diversity of Executive Functions and Their Contributions to Complex “Frontal Lobe” Tasks: A Latent Variable Analysis. Cognitive Psychology 41:49–100 Available at: http://dx.doi.org/10.1006/cogp.1999.0734.

Mohammad S, Turney P (2010) Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon. In: Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text, pp 26–34.

Mueller ST, Piper BJ (2014) The Psychology Experiment Building Language (PEBL) and PEBL Test Battery. J Neurosci Methods 222:250–259 Available at: http://dx.doi.org/10.1016/j.jneumeth.2013.10.024.

Oishi E (2006) Austin’s Speech Act Theory and the Speech Situation. Esercizi Filosofici 1:1– 14.

Oldfield RC (1971) The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9:97–113 Available at: http://dx.doi.org/10.1016/0028- 3932(71)90067-4.

Olszanowski M, Pochwatko G, Kuklinski K, Scibor-Rylski M, Lewinski P, Ohme RK (2015) Warsaw set of emotional facial expression pictures: a validation study of facial display photographs. Frontiers in Psychology 5 Available at: http://dx.doi.org/10.3389/fpsyg.2014.01516.

Panther K-U, Thornburg L (1998) A cognitive approach to inferencing in conversation. Journal of Pragmatics 30:755–769 Available at: Journal of Pragmatics A cognitive approach to inferencing in conversation☆ Author links open overlay panelKlaus- UwePanthera LindaThornburgb https://doi.org/10.1016/S0378-2166(98)00028-9.

Peirce JW (2007) PsychoPy—Psychophysics software in Python. Journal of Neuroscience Methods 162:8–13 Available at: http://dx.doi.org/10.1016/j.jneumeth.2006.11.017.

Phan KL, Wager T, Taylor SF, Liberzon I (2002) Functional neuroanatomy of emotion: a

27 meta-analysis of emotion activation studies in PET and fMRI. Neuroimage 16:331–348 Available at: http://dx.doi.org/10.1006/nimg.2002.1087.

Piper BJ, Mueller ST, Geerken AR, Dixon KL, Kroliczak G, Olsen RHJ, Miller JK (2015) Reliability and validity of neurobehavioral function on the Psychology Experimental Building Language test battery in young adults. PeerJ 3:e1460 Available at: http://dx.doi.org/10.7717/peerj.1460.

Rapp AM, Mutschler DE, Erb M (2012) Where in the brain is nonliteral language? A coordinate-based meta-analysis of functional magnetic resonance imaging studies. NeuroImage 63:600–610 Available at: http://dx.doi.org/10.1016/j.neuroimage.2012.06.022.

Rasgado-Toledo J, Lizcano-Cortés F, Olalde-Mathieu VE, Haquet GL, Zamora-Ursulo MA, Giordano M, Reyes-Aguilar A (2021) A dataset to study pragmatic language, and its underlying cognitive processes. Frontiers in Human Neuroscience 15:334 Available at: https://doi.org/10.3389/fnhum.2021.666210.

R Core Team (2020) R: A language and environment for statistical computing. R Foundation for Statistical Computing Available at: https://www.R-project.org/.

Reiman EM, Lane RD, Ahern GL, Schwartz GE, Davidson RJ, Friston KJ, Yun LS, Chen K (1997) Neuroanatomical correlates of externally and internally generated human emotion. Am J Psychiatry 154:918–925 Available at: http://dx.doi.org/10.1176/ajp.154.7.918.

Reyes-Aguilar A, Valles-Capetillo E, Giordano M (2018) A Quantitative Meta-analysis of Neuroimaging Studies of Pragmatic Language Comprehension: In Search of a Universal Neural Substrate. Neuroscience 395:60–88 Available at: http://dx.doi.org/10.1016/j.neuroscience.2018.10.043.

Reyes G (1995) El abecé de la pragmática. Arco Libros.

Schurz M, Radua J, Aichhorn M, Richlan F, Perner J (2014) Fractionating theory of mind: a meta-analysis of functional brain imaging studies. Neurosci Biobehav Rev 42:9–34 Available at: http://dx.doi.org/10.1016/j.neubiorev.2014.01.009.

Schurz M, Tholen MG, Perner J, Mars RB, Sallet J (2017) Specifying the brain anatomy underlying temporo-parietal junction activations for theory of mind: A review using probabilistic atlases from different imaging modalities. Hum Brain Mapp 38:4788–4805 Available at: http://dx.doi.org/10.1002/hbm.23675.

Schütz M, Ciaramidaro A, Martinelli A, Öller R, Hartmann D, Hein G, Iotzov V, Colle L, Becchio C, Walter H, Freitag CM (2020) Communicative intentions in autism spectrum disorder. Research in Autism Spectrum Disorders 79:101666 Available at: http://dx.doi.org/10.1016/j.rasd.2020.101666.

Spotorno N, Koun E, Prado J, Van Der Henst J-B, Noveck IA (2012) Neural evidence that utterance-processing entails mentalizing: the case of irony. Neuroimage 63:25–39 Available at: http://dx.doi.org/10.1016/j.neuroimage.2012.06.046.

Tettamanti M, Vaghi MM, Bara BG, Cappa SF, Enrici I, Adenzato M (2017) Effective

28 connectivity gateways to the Theory of Mind network in processing communicative intention. Neuroimage 155:169–176 Available at: http://dx.doi.org/10.1016/j.neuroimage.2017.04.050.

Van Dijk TA (1977) Context and cognition: Knowledge frames and speech act comprehension. Journal of Pragmatics 1:211–231 Available at: http://dx.doi.org/10.1016/0378-2166(77)90035-2.

Wagner P, Malisz Z, StefanKopp (2014) Gesture and speech in interaction: An overview. Speech Commun 57:209–232 Available at: http://dx.doi.org/10.1016/j.specom.2013.09.008.

Wechsler D (2012) WAIS-IV, Escala de inteligencia de Wechsler para adultos-IV: Manual de aplicación. Pearson Clinical & Talent Assessment. Available at: https://books.google.com/books/about/WAIS_IV.html?hl=&id=XBz2xwEACAAJ.

Westfall J, Kenny DA, Judd CM (2014) Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General 143:2020–2045 Available at: http://dx.doi.org/10.1037/xge0000014.

Yule G (2010) The Study of Language (Cambridge University Press, ed)., 4th ed. Cambridge University Press.