Adaptive empathy: A model for learning empathic responses based on feedback

Shamay-Tsoory SG1, 2 & Hertz U 2, 3

1 The Department of Psychology, University of Haifa

2Integrated Brain and Behavior Research Center (IBBRC), Haifa

3The Department of Cognitive Sciences, University of Haifa

Abstract

Empathy is usually deployed in social interactions. Nevertheless, common measures and examinations of empathy study this construct in isolation from the person in distress. In this paper we seek to extend the field of examination to include both empathizer and target in order to determine whether and how empathic responses are affected by feedback and learned through interaction. Building on computational approaches in feedback-based adaptations (e.g., no feedback, model-free and model-based learning), we propose a framework for understanding how empathic responses are learned based on feedback. In this framework, adaptive empathy, defined as the ability to adapt one’s empathic responses, is a central aspect of empathic skills, and can provide a new dimension to the evaluation and investigation of empathy. By extending existing neural models of empathy, we suggest that adaptive empathy may be mediated by interactions between the neural circuits associated with valuation, shared distress, observation-execution and mentalizing. Finally, we propose that adaptive empathy should be considered as a prominent facet of empathic capabilities with the potential to explain empathic behavior in health and in psychopathology.

1 Feedback-Based Adaptation as an Empathic Skill

Empathy is a multifaceted construct that encompasses both the cognitive and the emotional reactions of one individual (the empathizer) to the observed experiences of another (the target) (Decety and Jackson, 2004; Shamay-Tsoory, 2011). Behavioral and neuroimaging findings have led researchers to identify two broad types of empathic reactions. One is emotional empathy, which is characterized by feeling other people’s emotions. The other is cognitive empathy, which is characterized by understanding other people’s thoughts and motivations (Cuff et al., 2016; Gonzalez-Liencres et al., 2013; Smith, 2006). While definitions of emotional and cognitive empathy vary, the main objective of all empathic capabilities is clearly related to responding to the emotional state of another person. Empathy is thought to have evolved as a response that promotes distress reduction among group members (Decety et al., 2016; Eisenberg and Miller, 1987; Reeck et al., 2016; Telle and Pfister, 2016; Zaki, 2020; Zaki and Williams, 2013). Emphatic skills have been shown to play a major role in promoting well-being (Morelli et al., 2015), parenting (Atzil et al., 2018) and easing intergroup tensions (Influs et al., 2019). It has also been implicated in a variety of psychiatric disorders, such as depression, autistic spectrum disorder (ASD) ,schizophrenia, borderline personality disorder and psychopathy(Gonzalez-Liencres et al., 2013).

Despite a long tradition of studying empathy in social interactions in the fields of social (Stinson and Ickes, 1992; Thomas et al., 1997)and developmental(Hastings et al., 2000) psychology, the vast majority of paradigms testing empathy in neuroimaging and cognitive experiments involve passive observation of a suffering target. Typical studies of empathy evaluate a participant through a single reaction to pictures, questionnaires or vignettes depicting others in distress (e.g., Perry et al., 2011, Figure 1A) . Focusing on a solitary empathic response in isolation from the target has been useful in identifying different aspects of empathy. For example, this approach facilitated differentiating between a shared distress network activated during emotional empathy and a mentalizing network activated during cognitive empathy (Eres et al., 2015; Shamay-Tsoory, 2011). Here we suggest that as empathy is associated with relationships with others (Zaki, 2014), it must be examined in the context of dynamic interactions between empathizer and target over time. In such a context, an empathizer can detect the effects of his or her initial empathic response and adapt this response accordingly (Figure 1B). At the heart of our approach is framing empathy as a process that involves a feedback loop, where the likelihood to provide a specific empathic reaction changes during, and across, interactions based on feedback from the target (Main et al., 2017; Tamir and Thornton, 2018; Zaki et al., 2008) (Figure 2). For example, telling a distressed target to calm down may be helpful in diminishing distress for certain individuals, but for others such a response may backfire and make them more agitated. Indeed, in real-life social interactions empathy does not end when the empathizer perceives the target’s distress or deploys an empathic response (Main et al., 2017; Zaki et al., 2008).

2 The empathizer’s responses are received by the target and can cause changes in the target’s distress level. The empathizer then observes a consequent change in the target’s distress level, triggering a new cycle of empathic response. This cycle can span multiple instances of distress relief during the course of an interpersonal relationship (Goldstein et al., 2018) or a therapeutic relationship (Barrett- Lennard, 1981). Indeed, as evidenced by parental care, such a cycle can last a lifetime (Atzil et al., 2018). Therefore, we suggest that adaptive empathy, defined here as the ability to adjust one’s empathic responses appropriately during (online) and after (offline) social interactions, is a core aspect of empathy that should be investigated in addition to static empathic responses.

Figure 1: Illustration of empathic response Empathic responses in response to another's distress. In many experimental paradigms empathy is examined in isolation, in response to a set of static vignettes (A). However, in real life we are often face to face with the distressed person (B1) and can provide an empathic response to relieve the distress (B2). This live interaction allows us to detect the effect of our empathic response on the other person, and its effectiveness in relieving the distress, and use this information to adjust our current and future empathic responses, resulting in adaptive empathy. Illustration by Yali Ziv.

Here, we synthesize models of empathy with models of learning to propose an integrative neural model of adaptive empathy. Inspired by recent computational accounts of social behaviour (Lockwood et al., 2020; Lockwood and Klein-Flügge, 2020; Olsson et al., 2020; Tamir and Thornton, 2018; Zhang et al., 2020), we suggest three modes in which adaptive empathy can be ascribed for, namely open-loop, model-free and model-based adaptation. Within these modes we examine how this framework can be used to characterize different types of learners, e.g. rigid, flexible, and biased, and how one learner can display different adaptive empathy profiles across domains. Measures of adaptive empathy, we suggest, may enhance our understanding of the current prevalent static measures of cognitive and emotional aspects of empathy and propose applications of this approach for understanding psychopathology. Finally, we contend that developing new paradigms for assessing adaptive empathy can expand our understanding of empathy in real-life and encourage the development of novel computational and experimental approaches to investigate empathy.

3 Modes of adaptive learning - A computational approach to examine learning and adaptation in empathy

To understand and characterize empathy as an adaptive learning process, we adopt methods and frameworks from research in the fields of learning and control (Lockwood and Klein-Flügge, 2020). Computational approaches to learning and adaptation use such mechanisms as reinforcement learning, internal models and predictive coding to track trial-by-trial learning from responses, outcomes and feedback (Huang and Rao, 2011; Rescorla and Wagner, 1972; Sutton and Barto, 2012; Wolpert et al., 1995). These approaches are based on the notion that actions are selected based on their value and that the value of an action is updated by observing its outcome on a trial-by-trial basis. Actions become associated with outcomes through dynamic interactions with the environment, and these associations are used to guide policies for choosing actions that are most likely to lead to desirable outcomes (Daw et al., 2005). These models can therefore be used to characterize the way in which feedback from a distressed target, as well as contextual signals from the environment, can cause empathizers to adapt their initial empathic responses during empathic interaction.

Considering that empathy represents a set of reactions aimed at achieving distress regulation, it is suitable for characterization using a learning framework. Research on empathy indicates that it contains all the elements of a regulatory mechanism (Atzil et al., 2018; Heyes, 2018). An external event, someone else’s distress, signals a disturbance, a divergence from an optimal status, which needs to be regulated by triggering an emphatic response. Empathic responses can be covert (change in mood, emotions and thoughts), but are frequently overt (detectable facial or body expression, verbal responses) and are communicated back to the target. Importantly, the choice of response is guided by experience and expectation of its effectiveness in relieving the distress. After an empathic response has been deployed, feedback about the response’s effect is communicated in a similar manner: the target may show diminished distress, no effect or increased distress. On the basis of the target’s response the expected effectiveness of the response may be updated, and a new response is being deployed. This cycle can go on until distress is diminished, updating the expected effectiveness of emphatic responses for future interactions. The extent to which such a course of action takes place, the domains in which it operates, and the types of information it incorporates, lead to different modes of adaptive empathy – open-loop, model-free and model-based (Figure 2). In the following sections, we provide some detailed examples of situations in which adaptive empathy is deployed, drawing on the different computations underlying adaptation and learning.

4

Figure 2 – Three models of adaptive empathy

Adaptive empathy relies on transmission of distress signal from a target (light orange) to the empathizer (light blue). In open loop models, the distress signal triggers an empathic response, but this response is not shaped by feedback about the effectiveness of the response in relieving distress. In model-free feedback models the empathizer adapts her empathic response in line with incoming feedback about the target’s current level of distress. Model based feedback loop further enhances adaptive empathy by incorporating knowledge about the target’s beliefs, perspective, and internal state to select the appropriate empathic response.

Open-loop empathy

The simplest mode of adaptive empathy we consider is an open loop account. In this mode, no feedback-dependent learning takes place. That is, the empathizer is unlikely to change the expected effectiveness of empathic response after observing its effect on a target (Figure 2). In such cases, the empathic responses may have been triggered automatically by the distress signal, but do not change over the course of interaction. The initial response may be habitual, and automatically deployed which triggers a cascade of reactions (De Waal and Preston, 2017; Shamay-Tsoory, 2011) . Indeed, during affective sharing, i.e., sharing the physical pain of another, the target’s emotions automatically trigger a corresponding emotional state in the empathizer. The automatic responses elicited by distress cues from the environment may be due to associative learning between sensory cues and somatic responses (Heyes, 2018) and these responses can in turn prepare for actions (e.g. escape or prosocial responses). Automatic reactions such as mimicking (Jospe et al., 2018) are evident even when the target is not present. For example, we can share the pain of a victim we observe on the news even if we have no previous knowledge of this person and no chance of interacting with her in the future. Importantly, such automatic responses may be manifested as biases or priors in a model-free (or model-based)

5 mode of adaptive empathy, as we will discuss below (Figure 3). The hallmark of open-loop mode is the lack of adaptation, a rigidness in the deployment and update of empathic response when facing the target’s feedback.

Neuroimaging investigations of automatic reactions to the pain of others have revealed that in several brain areas responses to one’s own pain resemble one’s responses to the pain of others (Figure 4). Such studies show that the anterior insula (AI) and the dorsal-anterior/anterior-midcingulate cortex (dACC/aMCC) are activated in both experienced and observed pain (Bernhardt and Singer, 2012; Lamm et al., 2011) . This consistent response of the shared pain network suggests that it is not readily affected by feedback and will be triggered whenever a distress signal is apparent. Note, however, that this network may change and adapt over time, perhaps by means of top-down control from other networks. Nevertheless, it does not seem to exhibit the hallmark of online adaptation during interaction.

Closing the empathy loop: Model-free reinforcement learning

The main mode of adaptive empathy we discuss is a feedback loop model in which observations of the outcome of our empathic responses inform and affect our future empathic responses (Daw et al., 2005; Tamir and Thornton, 2018; Wolpert et al., 1995). Unlike the open-loop model, this model assumes that empathic responses are deployed based on their predicted outcome, i.e., on their expected ability to relieve the target’s distress (Figure 2). The outcome of each response is compared with its expected outcome, and any mismatch between expectation and outcome serves as a prediction error used to adapt future responses. In the case of an unexpected outcome, its value is updated. For example, when a reassuring response (e.g., handholding) does not lead to the expected reduction in distress, the response’s expected outcome is updated such that it is less likely to be deployed in the future. Such update may take place based on associative learning where the association between empathic response in a specific state, e.g. towards a specific person, and its distress relieve is learned over time. This version is referred to as model-free reinforcement learning, or feedback control (Daw et al., 2005; Wolpert et al., 2003).

One example of model-free feedback mechanism of adaptive empathy is synchronization. While the empathizer initially uses automatic mimicry in an open-loop manner (Jospe et al., 2018), over time the behavior of the empathizer becomes aligned and synchronized with that of the target. During synchronization, empathizers can adjust their body posture, tone of voice, and advice content to the target (Barrett-Lennard, 1981), thus potentially relieving the target’s distress (Kühn et al., 2011, 2010; Ramseyer and Tschacher, 2011). Building on models of social alignment it was suggested that, during social interactions, a predictive coding mechanism is geared to maximize alignment in movement, emotion, and cognitions with others (Shamay-Tsoory et al., 2019). This indicates that alignment with

6 others serves as the optimization principle, and achieving alignment reduces prediction errors, and makes the environment more predictable (Koban et al., 2019).

The observation-execution system, was shown to be engaged during such adaptive online interaction (Cook et al., 2014). In motor synchronization, continuous updating requires the existence of a system that observes the target’s behavior and automatically activates the empathizer’s own representations of this behavior. In turn, output from this shared representation automatically proceeds to motor areas of the brain where responses are prepared and executed (Preston and de Waal, 2002). The inferior frontal gyrus (IFG) as well as the inferior parietal lobe (IPL), which are frequently mentioned as playing a role in emotional empathy (Korisky et al., 2020; Shamay-Tsoory, 2011; Vachon-Presseau et al., 2012), also participate in observation-execution processes, among them the ability to observe and correspondingly execute specific actions. The IFG was found to participate both in synchronized finger tapping (Fairhurst et al., 2013) and in synchronous speaking (Jasmin et al., 2016). Research has demonstrated increased activity in the IPL during both motor (Cacioppo et al., 2014) and emotional (Korisky et al., 2020; Nummenmaa et al., 2008) synchronization. Synchronization is heavily dependent on constant observation and behavioral updating, including predictions of the other persons’ behavior (Wolpert et al., 2003). In addition to the role of the IFG in synchronization, Hein et al. (2015) reported activity in the IFG during learning of empathic responses. The authors show that this learning was driven by classical prediction errors, whose impact on empathy signals was mediated by an increase in positivity toward the out-group member. Notably, activation in the observation-execution system was reported to be coupled between individuals during synchrony. For example, evidence from hyperscanning fNIRS studies points to inter-brain coupling in the IFG during synchronized singing (Shimada et al., 2015). Likewise, studies using dual-EEG show inter-brain coupling in the alpha/mu band (8 to 12 or 13Hz) during synchronized class participation (Dikker et al., 2017). Inter-brain coupling in the alpha/mu band may represent activity in the observation- execution system (Astolfi et al., 2010; Dumas et al., 2010).

A model-free account of adaptive empathy can account for a number of behavioural patterns (Figure 3). To demonstrate these patterns we consider an adaptation of a two-armed bandit task, often used in learning and decision making studies (Hertz et al., 2018). In the social task, one is faced with a distressed target (Figure 1), and has to choose one of two empathic responses to deploy. One response leads to distress relieve, while the other does not. A learner may learn over time which empathic response is most efficient, increasing its expected relief value over time (Figure 3A). Learners may differ in the rate of learning, with some being very flexible and adapting quickly, while others are more rigid and display very little behavioral adaptation. Another distinction has to do with the learner’s initial response, which can be described as her prior on the effectiveness of the emphatic response, which biases her initial selection (Figure 3B). In such a case the initial bias in likelihood of

7 choosing the response may indicate such priors. Importantly, such initial bias may be the result of lifelong experience (Heyes, 2018), but unlike the open-loop account, here a change and adaptation in behavior occur as the response outcome is revealed. Finally, dynamic responses can differ within learners, as they may be more flexible when learning about one state-action pairs, for example learning in a person-specific manner (Farmer et al., 2019; Thornton et al., 2019), but slower to learn across other states (Figure 3C). For example, one may be slower to learn that a different response will be more effective for high intensity distress and another for low intensity distress (Shafir et al., 2016), than that a response may affect different people in a different way. Finally, it is possible that a mixture of these effects will be apparent, with people having both different learning rates and initial biases in different domains.

All these patterns can arise from using a single, and simple, learning model, by changing the values of parameters such as learning rate and starting condition (and decision noise which is not discussed here, but see (Daw, 2011; Findling et al., 2019)). It is therefore possible to use a simple experimental paradigm, where the states at hand are manipulated and a learning model is fitted to participants’ responses, to uncover the different adaptive empathy style and constraints displayed by an individual (Lockwood and Klein-Flügge, 2020). Such patterns may include overall slow or rigid adaptation, overall flexibility, or domain specific biases. These can be used to validate and enhance the current measures of empathy, emotional or cognitive, or reveal how previous experience and background may shape future relationships (Levy et al., 2016) by investigating biases in emphatic responses. In addition, extending the computational framework to more elaborate models of learning, for example Bayesian accounts of learning under uncertainty (Diaconescu et al., 2014; Siegel et al., 2018), may help characterize adaptive empathy by examining the way empathizers adapt their learning rate in face of volatile scenarios or persons.

There are a number of brain regions known to support associative learning, in non-social and social domains (Behrens et al., 2009; Olsson et al., 2020). Most notably, the ventral striatum (VS) was shown to track prediction errors in value based decisions (Delgado, 2007; Izuma et al., 2008; Schönberg et al., 2007), which is at the heart of the learning model described above. Social rewards were also shown to elicit such responses in the VS (Hertz et al., 2017; Klucharev et al., 2009; Wake and Izuma, 2017). Prediction error related to social behaviour of others were also associated with activity in other brain areas, such as the right Temporal-Parietal Junction (rTPJ) (Behrens et al., 2008; Bellucci et al., 2019; Koster-Hale and Saxe, 2013) and medial prefrontal cortex (MPFC) (Hertz et al., 2017). These brain areas, part of the brain social system, were associated with learning about others’ traits, while activity in the brain’s valuation system (including VS and ventromedial frontal cortex (vMPFC) (Lebreton et al., 2009) were associated with social rewards. In the case of adaptive empathy, the processing of emphatic response prediction errors may be displayed in both these

8 networks, as one learns about the response most effective to a specific person engaging the brain’s social system, but can also be processed in the brain’s valuation system, if relieving others’ distress is rewarding. Indeed, the balance in the engagement of such networks during emphatic response learning may be associated with individual differences in prosocial tendencies and motivations (Lockwood et al., 2017). The participation of the observation-execution system as well as reward-related signals may support a feedback-loop learning process, in which alignment serves as a learning signal leading to changes in empathic responses based on feedback (Shamay-Tsoory et al., 2019).

Figure 3 – Model-free adaptive empathy patterns

Adaptive empathy can rely on feedback based learning model, where an emphatic response is selected based on its expected effectiveness in reliving the target’s distress, and this effectiveness is learned based on the response outcome, using an associative mechanism. Such a model can give rise to a variety of learning patterns. (A) Differences in learning rates can lead to a more flexible pattern of adaptive empathy, where the learner adapts quickly to the target, to more rigid pattern of adaptation where learning rates are low. (B) Emphasizers can have some initial bias in their response selection (a prior) based on their previous experiences. These biases may change the starting point of the emphatic adaptation process, but can still leave room for adaptation as the interaction unfolds. (C) Adaptation may have different learning profile for different domains. For example, an emphasizer may learn quickly about the best emphatic response to a specific person, but fail to learn that a similar response is useful across similar distress intensities or in similar situations.

Expanding the loop: Model-based empathic learning

The model-free feedback-loop model of adaptive empathy did not require holding and updating a representation of the targets’ beliefs and perspective. It required learning about state-action pairs, where states could be different persons, different scenarios that triggered the distress (physical pain, disappointment, grieving, and so on), or different emotion intensity. However, during emphatic

9 interaction one may also use a representation of the other person’s perspective and believes (Den Ouden et al., 2005; Frith and Frith, 2003). To incorporate such information, we propose a model-based learning mode for adaptive empathy (Daw et al., 2005; Gershman et al., 2015; Wolpert et al., 2003) (Figure 2). A model-based attempt to diminish a target’s distress can therefore rely on previous knowledge of human behavior as well as on a mentalizing process in which mental states are inferred and attributed to specific persons. This knowledge, in turn, is applied when reacting to the behavior of others. This process, also known as theory of mind (ToM), has been shown to be affected by culture and developmental stage (Frith and Frith, 2003; Heyes and Frith, 2014). ToM requires high-order cognitive abilities such as cognitive flexibility (Decety and Jackson, 2004; Shamay-Tsoory et al., 2004) and episodic memory (Perry et al., 2011). In model-based learning, rather than simply updating the expected outcome of a response based on the current outcome and state, the learner also maintains an internal model of the target’s beliefs, traits and goals, and uses this model for adaptation. For example, a person may be distressed because he thinks he is more severely sick than he really is. Recognizing this gap may be useful in providing an emphatic response, either by not underestimating what the emphasizer knows is an exaggerated distress, and also by choosing a reaction that will help the target realize his distress is misplaced. A physician dismissing a patient by telling him he suffers from hysteria is failing to use model-based empathy, as well as a physician caving into his patient and subscribing him unnecessary treatment.

One example of an empathic response that relies on ToM is the recommendation of a regulation strategy for the target. Levy-Gigi and Shamay-Tsoory (2017) showed that regulation strategies chosen by the empathizer in the context of long-term romantic relationships were more effective at reducing distress than those selected by the target. Moreover, higher levels of trait cognitive empathy predicted successful regulation of the target’s emotions, suggesting that the ability to implement previous knowledge about the target is an important factor in distress relief. Borrowing on the two-step decision task (Sharp et al., 2016), commonly used to distinguish model-based and model-free learning, we can test whether empathizers use only the target’s expressed distress signal, and to what extent they incorporate knowledge about the target’s state when making their decision. For example, one can observe that when her toddler states that he is hungry, only about 70% of the times providing him with a snack will relive his distress, while sometimes he is finally being contended with a nap. This random pattern can be a source of frustration to her, as snack is the best response that can be inferred from the state and outcome – with association of 70%. However, she may realize that in many cases, 30% of the time, her son states that he is hungry when he is in fact tired. Incorporating this knowledge, his mother may be able to relive her son’s distress with a snack or a nap, according to her son’s activities during the day (Figure 4).

10

Figure 4 – Model-based adaptive empathy

Model free mode of adaptive empathy (right) associates expressed distress signals with the effectiveness of emphatic responses. An emphasizer may learn in this manner that when his toddler states that he is hungry, giving him a snack will relive his distress on 70% of the times (blue arrows), while on 30% of the times this response is not effective (red arrows), which makes it the most effective response most of the time. This will lead him to use the response ‘snack’ whenever his child says he is hungry. A model-based emphasizer (right) learned that her child’s inner states of ‘tired’ and ‘hungry’ sometimes (30% of the times) generate inappropriate distress statements (red arrow). By incorporating this knowledge, and her knowledge about events prior to the statement, she can recognize inappropriate statements and provide the most effective response. When her child states that he is hungry immediately after a snack, a model-based response will be a nap. When her child states that he is hungry after having a nap, the most effective response will be a snack.

Considering that model-based learning requires representing the mental states of others (e.g., beliefs, desires, intentions, and so on), this type of learning may be mediated by the mentalizing brain system (Abu-Akel and Shamay-Tsoory, 2011). The network associated with mentalization involves the medial prefrontal cortex (MPFC), the temporal lobe (TPJ), and the posterior superior temporal sulcus (pSTS) (Dulau, 2015; Shamay-Tsoory, 2011; Singer, 2006; Van Overwalle and Vandekerckhove, 2013). The brain areas implicated in mentalizing have also been shown to underlie tracking other people’s behavior and learning about others (Bellucci et al., 2019; Olsson et al., 2020). Likewise, Hallam et al.(2014) demonstrated that the left anterior temporal pole and the MPFC is active during a task of interpersonal emotion regulation. Evidence from strategic social games suggests that the prefrontal cortex, including the MPFC and the dorso-lateral prefrontal cortex (dlPFC), are engaged in building models of social interactions and relations (Hampton et al., 2008; Hertz et al., 2017; Kumaran et al., 2016), and their activity represent depth of strategic reasoning (Coricelli and Nagel, 2009; Yoshida et al., 2010). It therefore appears that the mentalization system may contribute to

11 choosing a regulation strategy, as it is necessary to understand the target’s mental state in order to try to change it (Zaki, 2020; Zaki and Williams, 2013).

A model of adaptive empathy

As we demonstrated above, we propose an adaptive empathy framework that considers three learning modes (Figure 2). When the empathizer observes someone in distress, an automatic distress mechanism, such as the shared pain network, is activated and can provide emphatic responses in an open-loop manner. State dependent responses, learned in a model-free manner through previous experiences, may also provide initial emphatic response based on expected effectiveness. Once feedback is available, an observation-execution system may participate, adjusting the empathic response according to the target’s response and updating the responses expected effectiveness according to the current state, e.g. person, emotion intensity, and environmental cues, in a model-free manner. In addition, the mentalizing system may come into play when the empathizer cognitively adopts the perspective of the other (Shamay-Tsoory, 2011). The mentalizing system may be used to incorporate knowledge about the target’s point of view, beliefs, and inner states, to inform the decision on the most effective emphatic response in in a model-based manner.

It is important to note that these modes of response are not mutually exclusive, and may operate one at a time or simultaneously, as suggested in the learning and cognitive control literature. For example, in the classic two-step decision task used to identify model-based and model-free decision-making patterns in humans (Daw et al., 2011), participants usually exhibit a mixture of model-based and model-free patterns. It is possible that cognitive and emotional empathic responses rely on open-loop patterns, model-free learning and model-based learning. The exact contribution of each may be flexible and appropriate to the task at hand. However, the examples given in figures 3 and 4 describe how experimental paradigms that can be useful in disentangling complex pattern of dynamic emphatic responses, and evaluate the contribution of different mode of response under different experimental conditions,

While several other neural networks may mediate adaptive empathy, the proposed framework immediately points to four candidate core brain networks that are associated with different computations supporting the different modes of adaptive empathy (Figure 5) that were previously associated with the different modes suggested above: a shared distress network that is automatically activated following detection of distress in the target (open-loop), an alignment system responsible for adjusting responses to the target over time (model-free), and a mentalizing network that tracks understanding the other’s mental state in a social context (model-free and model-based). The fourth system is the reward/valuation system which tracts and reacts to changes in the world, mostly those associated with rewards, which may track changes in the target’s distress based on feedback. The role

12 of the reward system may be to assign value to the empathizer’s reaction and therefore contribute to adapting the responses based on feedback from the target. However, it may be that other brain networks, such as the social brain network, may perform the same computation of change detection. These networks represent different computations associated with our framework, and future research may examine their role during emphatic interaction. One possibility is that the engagement of valuation system will be dependent on the social motivation of the empathizer, modulated by their prosocial tendencies (Crockett et al., 2017). It may also be that different networks will provide distinct behavioural effects, with observation-execution network providing fine-detail short term adaptations within an interaction, and mentalizing and social brain areas contributing to long term effects, carried from one interaction to the other.

Note that although we focus here on the brain networks that participate in empathy, it is highly likely that these same networks also mediate the target’s behavior. For example, the observation-execution network facilitates mutual alignment between the target and the empathizer. In addition, a person in distress may track the expected responses of a specific empathizer, for example, we may expect a close friend to hug us when we are in distress but be put off by similar behavior with a stranger.

Figure 3 – model of adaptive empathy Our proposed model incorporates four core brain networks which may underlie different components of adaptive empathy. Shared pain network (light yellow), including ACC and Insula, may operate as automatic distress detection, open-loop component. Observation-execution network (light green), including IFG and IPL, may underlie dynamic model-free adaptations, for example during imitation and synchronization. Mentalizing network (light blue), including MPFC and rTPJ, may track person specific responses’ effectiveness, and incorporate internal models of the person in distress to inform the empathic response in a model-based manner. Change detection in the valuation and reward system (light red), including the ventral striatum and

13 vMPFC, may detect the outcome of the selected empathic response, and send change signals to adapt future responses.

Out of the loop: Psychopathologies that involve impaired adaptive empathy

Beyond characterization of adaptive empathy dimension in the general population, the proposed adaptive empathy framework can be used to understand the social deficits presented in different psychopathologies in terms of how they are related to malfunctioning aspects of the adaptive process. Since empathy requires both emotional identification and emotional sharing (Bird and Viding, 2014), it could be the case that an individual is incapable of discerning changes in the target's emotion (e.g. Coll et al., 2017) and therefore would fail to learn how the adapt the empathic response. Similarly, variations in the tendency to learn from social feedback (Olsson et al., 2020)) or in adaptation of learning rates according to environmental or person specific volatility (Diaconescu et al., 2014; Siegel et al., 2018) could also account for poor adaptive empathy. One example, of a population which may exhibit difficulties in adaptive empathy is autism spectrum disorder (ASD), a condition that affects the way individuals interact with other people. Although research has repeatedly demonstrated that individuals with ASD exhibit aberrant empathy (e.g., Shamay-Tsoory et al., 2002), evidence regarding the exact empathy deficit found in ASD is conflicting. While individuals diagnosed with ASD have been shown to exhibit difficulties in mentalization (Baron-Cohen, 2002; Fletcher-Watson et al., 2014; Frith and Frith, 2003; Gaigg, 2012; Jones et al., 2010; Ponnet et al., 2004; Schwenck et al., 2012), some researchers have suggested that individuals with ASD exhibit preserved (e.g., Mazza et al., 2014) and even heightened (Adler et al., 2015) emotional empathy. Yet they have been shown to demonstrate poor performance on tasks that involve synchrony (Cheng et al., 2017; Feldman, 2007; Fitzpatrick et al., 2016; Marsh et al., 2013). The adaptive empathy approach may resolve these contradictory findings. It is possible that individuals with ASD have difficulties with adaptive empathy. Poor feedback-based learning capacities in the social domain may impair their ability to synchronize and mentalize, while their shared distress response remains intact. In line with this notion, Bolis and Schilbach (2017) suggested that people with ASD show difficulties in real-time interactions that require generation of adequate behavioral responses in the context of ongoing and reciprocal behavior. This implies that individuals with ASD may be capable of appropriately detecting the distress of others (open-loop), serving as input to the empathic processes. Yet they may have problems aligning (model-free) and selecting the correct response as well as problems incorporating and inferring the mental state of other people (model-based). Indeed, Rosenthal et al. (2019) recently characterized specific impairments in model-based mentalizing among ASD participants, showing that ASD participants demonstrated specific impairments in using their estimates of agent belief to understand agent intentions.

14 Examining model-based and model-free empathy in ASD may help lay the foundations for developing a new hypothesis to explain social deficits in ASD in terms of an adaptive empathy disorder. The clinical implication of such a hypothesis is that interventions for improving social deficits in ASD should focus on training individuals to react to feedback from a change in response.

Future directions: Adaptive empathy as an emphatic skill

An important implication of focusing on the impact of feedback on empathy is the notion that such adaptation of the empathic response is an important empathic skill. Indeed, adaptive empathy is crucial for social interactions as adaptation of empathic responses may help regulate a target’s distress by providing a more attuned and appropriate response (Figure 1). Future studies can demonstrate what types of response can be adapted, to what extent and for how long, and how this adaptive skill is related to trait empathy scales. Such studies can ultimately lead to new interventions that focus on training adaptive empathy.

We proposed here three modes are associated with different types of empathic computation, open- loop, model free and model based learning, which may rely on different neural networks and their interaction (Figure 5). Adaptive empathy may therefore include a variety of computational, cognitive, and neural features, and their combination may give rise to complex behaviors. We suggest that taking the dynamic dimension of empathy into account can enhance our understanding of the empathy construct, for example by examining the relationship between adaptive empathy and trait empathy, which is currently tracked using questionnaires and non-interactive tasks, and relate adaptive empathy with other social behaviors such as prosocial learning (Crockett et al., 2014; Lockwood et al., 2016). It may also help construct a more ecological experiments, examining how empathy may operate in real-life interactions. Finally, such tasks can also be used to characterize specific deficits in clinical populations, adding a new dimension to the assessment and description of psychopathologies such as ASD.

Here we focused on modes of learning and adaptation, and provide a mechanistic description of the different ways in which such process can take place. It is important to note, that we do not address other important aspects of empathy, and social learning in general, such as social motivation and social perception. Motivation plays an important role in the decision to engage and provide an empathic response (Cameron et al., 2019; Hughes et al., 2017; Mccall and Singer, 2012). Indeed, engaging in an emphatic interaction can be taxing (Coetzee and Klopper, 2010), and sometimes the role of empathy is to realize when not to engage. Detection of distress signals is also crucial in the triggering of emphatic process, making the two concepts confused in some cases (Coll et al., 2017), and problems in detection of emotional signals can lead to deficits in empathic behaviour, while

15 enhancing social saliency may result in increased empathy (Abu-Akel et al., 2014; Iria and Barbosa, 2009; Shamay-Tsoory and Abu-Akel, 2016). While both these concepts, and many others, are important to describe engagement in empathic interaction, they fall beyond our current scope. We show that even when assuming an intact ability to detect distress signal and contextual information, and motivation to engage in an emphatic interaction, different modes of adaptive empathy may lead to distinct behavioural patterns.

Finally, we encourage the examination of the relationship between adaptive empathy and trait empathy, which is currently tracked using questionnaires and non-interactive tasks. Measures of trait emotional empathy may be used to predict model-free learning, while measures of trait cognitive empathy can predict model-based adaptive empathy. This line of studies should incorporate the well- known properties of trait and situational empathy that dominate the literature and also include novel findings regarding adaptive empathy skills. This approach may provide a much-needed new dimension of empathy to explain how this construct operates in real-life social interactions.

16 References

Abu-Akel, A., Palgi, S., Klein, E., Decety, J., Shamay-Tsoory, S., 2014. Oxytocin increases empathy to pain when adopting the other- but not the self-perspective. Soc. Neurosci. 1–9. https://doi.org/10.1080/17470919.2014.948637

Abu-Akel, A., Shamay-Tsoory, S., 2011. Neuroanatomical and neurochemical bases of theory of mind. Neuropsychologia 49, 2971–2984. https://doi.org/10.1016/j.neuropsychologia.2011.07.012

Adler, N., Dvash, J., Shamay-Tsoory, S.G., 2015. Empathic Embarrassment Accuracy in Autism Spectrum Disorder. Autism Res. 8, 241–249. https://doi.org/10.1002/aur.1439

Astolfi, L., Cincotti, F., Mattia, D., De Vico Fallani, F., Salinari, S., Vecchiato, G., Toppi, J., Wilke, C., Doud, A., Yuan, H., He, B., Babiloni, F., 2010. Simultaneous estimation of cortical activity during social interactions by using EEG hyperscannings, in: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC’10. IEEE, pp. 2814–2817. https://doi.org/10.1109/IEMBS.2010.5626555

Atzil, S., Gao, W., Fradkin, I., Barrett, L.F., 2018. Growing a social brain. Nat. Hum. Behav. 2, 624– 636. https://doi.org/10.1038/s41562-018-0384-6

Baron-Cohen, S., 2002. The extreme male brain theory of autism. Trends Cogn. Sci. 6, 248–254. https://doi.org/10.1016/S1364-6613(02)01904-6

Barrett-Lennard, G.T., 1981. The empathy cycle: Refinement of a nuclear concept. J. Couns. Psychol. 28, 91–100. https://doi.org/10.1037/0022-0167.28.2.91

Behrens, T.E.J., Hunt, L.T., Rushworth, M.F.S., 2009. The computation of social behavior. Science 324, 1160–4. https://doi.org/10.1126/science.1169694

Behrens, T.E.J., Hunt, L.T., Woolrich, M.W., Rushworth, M.F.S., 2008. Associative learning of social value. Nature 456, 245–9. https://doi.org/10.1038/nature07538

Bellucci, G., Molter, F., Park, S.Q., 2019. Neural representations of honesty predict future trust behavior. Nat. Commun. 10, 5184. https://doi.org/10.1038/s41467-019-13261-8

Bernhardt, B.C., Singer, T., 2012. The Neural Basis of Empathy. Annu. Rev. Neurosci. 35, 1–23. https://doi.org/10.1146/annurev-neuro-062111-150536

Bird, G., Viding, E., 2014. The self to other model of empathy: Providing a new framework for understanding empathy impairments in psychopathy, autism, and alexithymia. Neurosci. Biobehav. Rev. 47, 520–532. https://doi.org/10.1016/j.neubiorev.2014.09.021

Bolis, D., Schilbach, L., 2017. Beyond one Bayesian brain: Modeling intra- and inter-personal processes during social interaction: Commentary on “Mentalizing homeostasis: The social origins of interoceptive inference” by Fotopoulou & Tsakiris. Neuropsychoanalysis 19, 35–38.

17 https://doi.org/10.1080/15294145.2017.1295215

Cacioppo, S., Zhou, H., Monteleone, G., Majka, E.A., Quinn, K.A., Ball, A.B., Norman, G.J., Semin, G.R., Cacioppo, J.T., 2014. You are in sync with me: Neural correlates of interpersonal synchrony with a partner. Neuroscience 277, 842–858. https://doi.org/10.1016/j.neuroscience.2014.07.051

Cameron, C.D., Hutcherson, C.A., Ferguson, A.M., Scheffer, J.A., Hadjiandreou, E., Inzlicht, M., 2019. Empathy is hard work: People choose to avoid empathy because of its cognitive costs. J. Exp. Psychol. Gen. 148, 962–976. https://doi.org/10.1037/xge0000595

Cheng, M., Kato, M., Tseng, C., 2017. Gender and autistic traits modulate implicit motor synchrony. PLoS One.

Coetzee, S.K., Klopper, H.C., 2010. Compassion fatigue within nursing practice: A concept analysis. Nurs. Health Sci. 12, 235–243. https://doi.org/10.1111/j.1442-2018.2010.00526.x

Coll, M.P., Viding, E., Rütgen, M., Silani, G., Lamm, C., Catmur, C., Bird, G., 2017. Are we really measuring empathy? Proposal for a new measurement framework. Neurosci. Biobehav. Rev. 83, 132–139. https://doi.org/10.1016/j.neubiorev.2017.10.009

Cook, R., Bird, G., Catmur, C., Press, C., Heyes, C., 2014. Mirror neurons: From origin to function. Behav. Brain Sci. 37, 177–192. https://doi.org/10.1017/S0140525X13000903

Coricelli, G., Nagel, R., 2009. Neural correlates of depth of strategic reasoning in medial prefrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 106, 9163–9168. https://doi.org/10.1073/pnas.0807721106

Crockett, M.J., Kurth-Nelson, Z., Siegel, J.Z., Dayan, P., Dolan, R.J., 2014. Harm to others outweighs harm to self in moral decision making. Proc. Natl. Acad. Sci. 111, 17320–17325. https://doi.org/10.1073/pnas.1408988111

Crockett, M.J., Siegel, J.Z., Kurth-Nelson, Z., Dayan, P., Dolan, R.J., 2017. Moral transgressions corrupt neural representations of value. Nat. Neurosci. 20, 879–885. https://doi.org/10.1038/nn.4557

Cuff, B.M.P., Brown, S.J., Taylor, L., Howat, D.J., 2016. Empathy: A Review of the Concept. Emot. Rev. 8, 144–153. https://doi.org/10.1177/1754073914558466

Daw, N.D., 2011. Trial-by-trial data analysis using computational models, in: Phelps, E.A., Robbins, T.W., Delgado, M. (Eds.), Decision Making, Affect, and Learning. Oxford University Press, New York, New York, USA, pp. 3–38. https://doi.org/10.1093/acprof:oso/9780199600434.003.0001

Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P., Dolan, R.J., 2011. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–15. https://doi.org/10.1016/j.neuron.2011.02.027

Daw, N.D., Niv, Y., Dayan, P., 2005. Uncertainty-based competition between prefrontal and

18 dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–11. https://doi.org/10.1038/nn1560

De Waal, F.B.M., Preston, S.D., 2017. Mammalian empathy: Behavioural manifestations and neural basis. Nat. Rev. Neurosci. 18, 498–509. https://doi.org/10.1038/nrn.2017.72

Decety, J., Bartal, I.B.A., Uzefovsky, F., Knafo-Noam, A., 2016. Empathy as a driver of prosocial behaviour: Highly conserved neurobehavioural mechanisms across species. Philos. Trans. R. Soc. B Biol. Sci. 371. https://doi.org/10.1098/rstb.2015.0077

Decety, J., Jackson, P.L., 2004. The Functional Architecture of Human Empathy. Behav. Cogn. Neurosci. Rev. 3, 71–100. https://doi.org/10.1177/1534582304267187

Delgado, M.R., 2007. Reward-related responses in the human striatum. Ann. N. Y. Acad. Sci. 1104, 70–88. https://doi.org/10.1196/annals.1390.002

Den Ouden, H.E.M., Frith, U., Frith, C., Blakemore, S.J., 2005. Thinking about intentions. Neuroimage 28, 787–796. https://doi.org/10.1016/j.neuroimage.2005.05.001

Diaconescu, A.O., Mathys, C., Weber, L. a E., Daunizeau, J., Kasper, L., Lomakina, E.I., Fehr, E., Stephan, K.E., 2014. Inferring on the intentions of others by hierarchical bayesian learning. PLoS Comput. Biol. 10, e1003810. https://doi.org/10.1371/journal.pcbi.1003810

Dikker, S., Wan, L., Davidesco, I., Kaggen, L., Oostrik, M., McClintock, J., Rowland, J., Michalareas, G., Van Bavel, J.J., Ding, M., Poeppel, D., 2017. Brain-to-Brain Synchrony Tracks Real-World Dynamic Group Interactions in the Classroom. Curr. Biol. 27, 1375–1380. https://doi.org/10.1016/j.cub.2017.04.002

Dulau, C., 2015. Introduction to Social Cognition. Springer, Cham, pp. 181–194. https://doi.org/10.1007/978-3-319-18464-7_13

Dumas, G., Nadel, J., Soussignan, R., Martinerie, J., Garnero, L., 2010. Inter-brain synchronization during social interaction. PLoS One 5, e12166. https://doi.org/10.1371/journal.pone.0012166

Eisenberg, N., Miller, P.A., 1987. The Relation of Empathy to Prosocial and Related Behaviors. Psychol. Bull. https://doi.org/10.1037/0033-2909.101.1.91

Eres, R., Decety, J., Louis, W.R., Molenberghs, P., 2015. Individual differences in local gray matter density are associated with differences in affective and cognitive empathy. Neuroimage 117, 305– 310. https://doi.org/10.1016/j.neuroimage.2015.05.038

Fairhurst, M.T., Janata, P., Keller, P.E., 2013. Being and Feeling in Sync with an Adaptive Virtual Partner: Brain Mechanisms Underlying Dynamic Cooperativity. Cereb. Cortex 23, 2592–2600. https://doi.org/10.1093/cercor/bhs243

Farmer, H., Hertz, U., Hamilton, A., 2019. The Neural Basis of Shared Preference Learning. Soc. Cogn.

19 Affect. Neurosci. https://doi.org/10.1093/scan/nsz076

Feldman, R., 2007. Parent-infant synchrony: Biological foundations and developmental outcomes. Curr. Dir. Psychol. Sci. 16, 340–345. https://doi.org/10.1111/j.1467-8721.2007.00532.x

Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., Wyart, V., 2019. Computational noise in reward-guided learning drives behavioral variability in volatile environments. Nat. Neurosci. 22, 2066–2077. https://doi.org/10.1038/s41593-019-0518-9

Fitzpatrick, P., Frazier, J.A., Cochran, D.M., Mitchell, T., Coleman, C., Schmidt, R.C., 2016. Impairments of Social Motor Synchrony Evident in Autism Spectrum Disorder. Front. Psychol. 7, 1323. https://doi.org/10.3389/fpsyg.2016.01323

Fletcher-Watson, S., McConnell, F., Manola, E., McConachie, H., 2014. Interventions based on the Theory of Mind cognitive model for autism spectrum disorder (ASD). Cochrane Database Syst. Rev. 2014. https://doi.org/10.1002/14651858.CD008785.pub2

Frith, U., Frith, C.D., 2003. Development and neurophysiology of mentalizing. Philos. Trans. R. Soc. London. Ser. B Biol. Sci. 358, 459–473. https://doi.org/10.1098/rstb.2002.1218

Gaigg, S.B., 2012. The interplay between emotion and cognition in Autism Spectrum Disorder: Implications for developmental theory. Front. Integr. Neurosci. 6, 113. https://doi.org/10.3389/fnint.2012.00113

Gershman, S.J., Norman, K.A., Niv, Y., 2015. Discovering latent causes in reinforcement learning. Curr. Opin. Behav. Sci. 5, 43–50. https://doi.org/10.1016/j.cobeha.2015.07.007

Goldstein, P., Weissman-Fogel, I., Dumas, G., Shamay-Tsoory, S.G., 2018. Brain-to-brain coupling during handholding is associated with pain reduction. Proc. Natl. Acad. Sci. 115, E2528–E2537. https://doi.org/10.1073/pnas.1703643115

Gonzalez-Liencres, C., Shamay-Tsoory, S.G., Brüne, M., 2013. Towards a neuroscience of empathy: Ontogeny, phylogeny, brain mechanisms, context and psychopathology. Neurosci. Biobehav. Rev. 37, 1537–1548. https://doi.org/10.1016/j.neubiorev.2013.05.001

Hallam, G.P., Webb, T.L., Sheeran, P., Miles, E., Niven, K., Wilkinson, I.D., Hunter, M.D., Woodruff, P.W.R., Totterdell, P., Farrow, T.F.D., 2014. The neural correlates of regulating another person’s emotions: An exploratory fMRI study. Front. Hum. Neurosci. 8. https://doi.org/10.3389/fnhum.2014.00376

Hampton, A.N., Bossaerts, P., O’Doherty, J.P., 2008. Neural correlates of mentalizing-related computations during strategic interactions in humans. Proc. Natl. Acad. Sci. 105, 6741–6746. https://doi.org/10.1073/pnas.0711099105

Hastings, P.D., Zahn-Waxler, C., Robinson, J., Usher, B., Bridges, D., 2000. The development of

20 concern for others in children with behavior problems. Dev. Psychol. 36, 531–546. https://doi.org/10.1037/0012-1649.36.5.531

Hertz, U., Bahrami, B., Keramati, M., 2018. Stochastic satisficing account of confidence in uncertain value-based decisions. PLoS One 13, e0195399. https://doi.org/10.1371/journal.pone.0195399

Hertz, U., Palminteri, S., Brunetti, S., Olesen, C., Frith, C.D., Bahrami, B., 2017. Neural computations underpinning the strategic management of influence in advice giving. Nat. Commun. 8, 2191. https://doi.org/10.1038/s41467-017-02314-5

Heyes, C., 2018. Empathy is not in our genes. Neurosci. Biobehav. Rev. 95, 499–507. https://doi.org/10.1016/j.neubiorev.2018.11.001

Heyes, C.M., Frith, C.D., 2014. The cultural evolution of mind reading. Science (80-. ). 344, 1243091– 1243091. https://doi.org/10.1126/science.1243091

Huang, Y., Rao, R.P.N.N., 2011. Predictive coding. Wiley Interdiscip. Rev. Cogn. Sci. 2, 580–593. https://doi.org/10.1002/wcs.142

Hughes, B.L., Zaki, J., Ambady, N., 2017. Motivation alters impression formation and related neural systems. Soc. Cogn. Affect. Neurosci. 12, 49–60. https://doi.org/10.1093/scan/nsw147

Influs, M., Pratt, M., Masalha, S., Zagoory-Sharon, O., Feldman, R., 2019. A social neuroscience approach to conflict resolution: Dialogue intervention to Israeli and Palestinian youth impacts oxytocin and empathy. Soc. Neurosci. 14, 378–389. https://doi.org/10.1080/17470919.2018.1479983

Iria, C., Barbosa, F., 2009. Perception of facial expressions of fear: Comparative research with criminal and non-criminal psychopaths. J. Forensic Psychiatry Psychol. 20, 66–73. https://doi.org/10.1080/14789940802214218

Izuma, K., Saito, D.N., Sadato, N., 2008. Processing of social and monetary rewards in the human striatum. Neuron 58, 284–94. https://doi.org/10.1016/j.neuron.2008.03.020

Jasmin, K.M., McGettigan, C., Agnew, Z.K., Lavan, N., Josephs, O., Cummins, F., Scott, S.K., 2016. Cohesion and Joint Speech: Right Hemisphere Contributions to Synchronized Vocal Production. J. Neurosci. 36, 4669–4680. https://doi.org/10.1523/JNEUROSCI.4075-15.2016

Jones, A.P., Happé, F.G.E., Gilbert, F., Burnett, S., Viding, E., 2010. Feeling, caring, knowing: different types of empathy deficit in boys with psychopathic tendencies and autism spectrum disorder. J. Child Psychol. Psychiatry 51, 1188–1197. https://doi.org/10.1111/j.1469-7610.2010.02280.x

Jospe, K., Flöe, A., Lavidor, M., 2018. The interaction between embodiment and empathy in facial expression recognition. Soc. Cogn. Affect. Neurosci. 13, 203–215. https://doi.org/10.1093/scan/nsy005

21 Klucharev, V., Hytönen, K., Rijpkema, M., Smidts, A., Fernández, G., 2009. Reinforcement Learning Signal Predicts Social Conformity. Neuron 61, 140–151. https://doi.org/10.1016/j.neuron.2008.11.027

Koban, L., Ramamoorthy, A., Konvalinka, I., 2019. Why do we fall into sync with others? Interpersonal synchronization and the brain’s optimization principle. Soc. Neurosci. 14, 1–9. https://doi.org/10.1080/17470919.2017.1400463

Korisky, A., Eisenberger, N.I., Nevat, M., Weissman-Fogel, I., Shamay-Tsoory, S.G., 2020. A dual- brain approach for understanding the neuralmechanisms that underlie the comforting effects of social touch. Cortex 127, 333–346. https://doi.org/10.1016/j.cortex.2020.01.028

Koster-Hale, J., Saxe, R., 2013. Theory of Mind: A Neural Prediction Problem. Neuron 79, 836–848. https://doi.org/10.1016/j.neuron.2013.08.020

Kühn, S., Müller, B.C.N., van Baaren, R.B., Wietzker, A., Dijksterhuis, A., Brass, M., 2010. Why do I like you when you behave like me? Neural mechanisms mediating positive consequences of observing someone being imitated. Soc. Neurosci. 5, 384–392. https://doi.org/10.1080/17470911003633750

Kühn, S., Müller, B.C.N., van der Leij, A., Dijksterhuis, A., Brass, M., van Baaren, R.B., 2011. Neural correlates of emotional synchrony. Soc. Cogn. Affect. Neurosci. 6, 368–374. https://doi.org/10.1093/scan/nsq044

Kumaran, D., Banino, A., Blundell, C., Hassabis, D., Dayan, P., 2016. Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information. Neuron 92, 1135–1147. https://doi.org/10.1016/j.neuron.2016.10.052

Lamm, C., Decety, J., Singer, T., 2011. Meta-analytic evidence for common and distinct neural networks associated with directly experienced pain and empathy for pain. Neuroimage 54, 2492– 2502. https://doi.org/10.1016/j.neuroimage.2010.10.014

Lebreton, M., Jorge, S., Michel, V., Thirion, B., Pessiglione, M., 2009. An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging. Neuron 64, 431–439. https://doi.org/10.1016/j.neuron.2009.09.040

Levy-Gigi, E., Shamay-Tsoory, S.G., 2017. Help me if you can: Evaluating the effectiveness of interpersonal compared to intrapersonal emotion regulation in reducing distress. J. Behav. Ther. Exp. Psychiatry 55, 33–40. https://doi.org/10.1016/j.jbtep.2016.11.008

Levy, J., Goldstein, A., Influs, M., Masalha, S., Zagoory-Sharon, O., Feldman, R., 2016. Adolescents growing up amidst intractable conflict attenuate brain response to pain of outgroup. Proc. Natl. Acad. Sci. U. S. A. 113, 13696–13701. https://doi.org/10.1073/pnas.1612903113

Lockwood, P.L., Apps, M.A.J., Chang, S.W.C., 2020. Is There a ‘Social’ Brain? Implementations and

22 Algorithms. Trends Cogn. Sci. 24, 802–813. https://doi.org/10.1016/j.tics.2020.06.011

Lockwood, P.L., Apps, M.A.J., Valton, V., Viding, E., Roiser, J.P., 2016. Neurocomputational mechanisms of prosocial learning and links to empathy. Proc. Natl. Acad. Sci. 113, 9763–9768. https://doi.org/10.1073/pnas.1603198113

Lockwood, P.L., Hamonet, M., Zhang, S.H., Ratnavel, A., Salmony, F.U., Husain, M., Apps, M.A.J., 2017. Prosocial apathy for helping others when effort is required. Nat. Hum. Behav. 1, 1–10. https://doi.org/10.1038/s41562-017-0131

Lockwood, P.L., Klein-Flügge, M.C., 2020. Computational modelling of social cognition and behaviour—a reinforcement learning primer. Soc. Cogn. Affect. Neurosci. 1–11. https://doi.org/10.1093/scan/nsaa040

Main, A., Walle, E.A., Kho, C., Halpern, J., 2017. The Interpersonal Functions of Empathy: A Relational Perspective. Emot. Rev. 9, 358–366. https://doi.org/10.1177/1754073916669440

Marsh, K.L., Isenhower, R.W., Richardson, M.J., Helt, M., Verbalis, A.D., Schmidt, R.C., Fein, D., 2013. Autism and social disconnection in interpersonal rocking. Front. Integr. Neurosci. 7, 4. https://doi.org/10.3389/fnint.2013.00004

Mazza, M., Pino, M.C., Mariano, M., Tempesta, D., Ferrara, M., De Berardis, D., Masedu, F., Valenti, M., 2014. Affective and cognitive empathy in adolescents with autism spectrum disorder. Front. Hum. Neurosci. 8, 791. https://doi.org/10.3389/fnhum.2014.00791

Mccall, C., Singer, T., 2012. The animal and human neuroendocrinology of social cognition, motivation and behavior. Nat. Neurosci. 15, 681–688. https://doi.org/10.1038/nn.3084

Morelli, S.A., Lieberman, M.D., Zaki, J., 2015. The Emerging Study of Positive Empathy. Soc. Personal. Psychol. Compass 9, 57–68. https://doi.org/10.1111/spc3.12157

Nummenmaa, L., Hirvonen, J., Parkkola, R., Hietanen, J.K., 2008. Is emotional contagion special? An fMRI study on neural systems for affective and cognitive empathy. Neuroimage 43, 571–580. https://doi.org/10.1016/j.neuroimage.2008.08.014

Olsson, A., Knapska, E., Lindström, B., 2020. The neural and computational systems of social learning. Nat. Rev. Neurosci. 21, 197–212. https://doi.org/10.1038/s41583-020-0276-4

Perry, D., Hendler, T., Shamay-Tsoory, S.G., 2011. Projecting memories: The role of the hippocampus in emotional mentalizing. Neuroimage 54, 1669–1676. https://doi.org/10.1016/j.neuroimage.2010.08.057

Ponnet, K.S., Roeyers, H., Buysse, A., De Clercq, A., Van Der Heyden, E., 2004. Advanced Mind- Reading in Adults with Asperger Syndrome. Autism 8, 249–266. https://doi.org/10.1177/1362361304045214

23 Preston, S.D., de Waal, F.B.M., 2002. Empathy: Its ultimate and proximate bases. Behav. Brain Sci. 25, 1–20. https://doi.org/10.1017/S0140525X02000018

Ramseyer, F., Tschacher, W., 2011. Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and outcome. J. Consult. Clin. Psychol. 79, 284–295. https://doi.org/10.1037/a0023419

Reeck, C., Ames, D.R., Ochsner, K.N., 2016. The Social Regulation of Emotion: An Integrative, Cross- Disciplinary Model. Trends Cogn. Sci. 20, 47–63. https://doi.org/10.1016/j.tics.2015.09.003

Rescorla, R.A., Wagner, A.R., 1972. A Theory of Pavlovian Conditioning : Variations in the Effectiveness of Reinforcement and Nonreinforcement, in: Black, A., Prokasy, W.F. (Eds.), Classical Conditioning II: Current Research and Theory. Appleton-Century-Crofts, New York, New York, USA, pp. 64–99.

Rosenthal, I.A., Hutcherson, C.A., Adolphs, R., Stanley, D.A., 2019. Deconstructing Theory-of-Mind Impairment in High-Functioning Adults with Autism. Curr. Biol. 29, 513-519.e6. https://doi.org/10.1016/j.cub.2018.12.039

Schönberg, T., Daw, N.D., Joel, D., O’Doherty, J.P., 2007. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–7. https://doi.org/10.1523/JNEUROSCI.2496-07.2007

Schwenck, C., Mergenthaler, J., Keller, K., Zech, J., Salehi, S., Taurines, R., Romanos, M., Schecklmann, M., Schneider, W., Warnke, A., Freitag, C.M., 2012. Empathy in children with autism and conduct disorder: group-specific profiles and developmental aspects. J. Child Psychol. Psychiatry 53, 651–659. https://doi.org/10.1111/j.1469-7610.2011.02499.x

Shafir, R., Thiruchselvam, R., Suri, G., Gross, J.J., Sheppes, G., 2016. Neural processing of emotional- intensity predicts emotion regulation choice. Soc. Cogn. Affect. Neurosci. 11, 1863–1871. https://doi.org/10.1093/scan/nsw114

Shamay-Tsoory, S.G., 2011. The Neural Bases for Empathy. Neurosci. 17, 18–24. https://doi.org/10.1177/1073858410379268

Shamay-Tsoory, S.G., Abu-Akel, A., 2016. The Social Salience Hypothesis of Oxytocin. Biol. Psychiatry 79, 194–202. https://doi.org/10.1016/j.biopsych.2015.07.020

Shamay-Tsoory, S.G., Abu-Akel, A., Palgi, S., Sulieman, R., Fischer-Shofty, M., Levkovitz, Y., Decety, J., 2013. Giving peace a chance: Oxytocin increases empathy to pain in the context of the Israeli-Palestinian conflict. Psychoneuroendocrinology 38, 3139–3144. https://doi.org/10.1016/j.psyneuen.2013.09.015

Shamay-Tsoory, S.G., Saporta, N., Marton-Alper, I.Z., Gvirts, H.Z., 2019. Herding Brains: A Core Neural Mechanism for Social Alignment. Trends Cogn. Sci. 23, 174–186.

24 https://doi.org/10.1016/j.tics.2019.01.002

Shamay-Tsoory, S.G., Tomer, R., Goldsher, D., Berger, B.D., Aharon-Peretz, J., 2004. Impairment in cognitive and affective empathy in patients with brain lesions: Anatomical and cognitive correlates. J. Clin. Exp. Neuropsychol. 26, 1113–1127. https://doi.org/10.1080/13803390490515531

Shamay-Tsoory, S.G., Tomer, R., Yaniv, S., Aharon-Peretz, J., 2002. Empathy deficits in Asperger syndrome: a cognitive profile. Neurocase 8, 245–52. https://doi.org/10.1093/neucas/8.3.245

Sharp, M.E., Foerde, K., Daw, N.D., Shohamy, D., 2016. Dopamine selectively remediates “model- based” reward learning: A computational approach. Brain 139, 355–364. https://doi.org/10.1093/brain/awv347

Shimada, Y.M., Osaka, M., Azuma, M., Yaoi, K., Minamoto, T., Osaka, N., 2015. How Two Brains Make One Synchronized Mind in the Inferior Frontal Cortex: fNIRS-Based Hyperscanning During Cooperative Singing. Front. Psychol. 6, 1811. https://doi.org/10.3389/fpsyg.2015.01811

Siegel, J.Z., Mathys, C., Rutledge, R.B., Crockett, M.J., 2018. Beliefs about bad people are volatile. Nat. Hum. Behav. 2, 750–756. https://doi.org/10.1038/s41562-018-0425-1

Singer, T., 2006. The neuronal basis and ontogeny of empathy and mind reading: Review of literature and implications for future research. Neurosci. Biobehav. Rev. 30, 855–863. https://doi.org/10.1016/j.neubiorev.2006.06.011

Smith, A., 2006. Cognitive Empathy and Emotional Empathy in Human Behavior and Evolution. Psychol. Rec. 56, 3–21. https://doi.org/10.1007/BF03395534

Stinson, L., Ickes, W., 1992. Empathic accuracy in the interactions of male friends versus male strangers. J. Pers. Soc. Psychol. 62, 787–797. https://doi.org/10.1037/0022-3514.62.5.787

Sutton, R.S., Barto, A.G., 2012. Reinforcement learning: An introduction, 2nd ed. MIT press, Cambridge. https://doi.org/10.1109/MED.2013.6608833

Tamir, D.I., Thornton, M.A., 2018. Modeling the Predictive Social Mind. Trends Cogn. Sci. xx, 201– 212. https://doi.org/10.1016/j.tics.2017.12.005

Telle, N.-T.T., Pfister, H.-R.R., 2016. Positive Empathy and Prosocial Behavior: A Neglected Link. Emot. Rev. 8, 154–163. https://doi.org/10.1177/1754073915586817

Thomas, G., Fletcher, G.J.O., Lange, C., 1997. On-line empathic accuracy in marital interaction. J. Pers. Soc. Psychol. 72, 839–850. https://doi.org/10.1037/0022-3514.72.4.839

Thornton, M.A., Weaverdyck, M.E., Tamir, D.I., 2019. The brain represents people as the mental states they habitually experience. Nat. Commun. 10, 2291. https://doi.org/10.1038/s41467-019-10309- 7

25 Vachon-Presseau, E., Roy, M., Martel, M.O., Albouy, G., Chen, J., Budell, L., Sullivan, M.J., Jackson, P.L., Rainville, P., 2012. Neural processing of sensory and emotional-communicative information associated with the perception of vicarious pain. Neuroimage 63, 54–62. https://doi.org/https://doi.org/10.1016/j.neuroimage.2012.06.030

Van Overwalle, F., Vandekerckhove, M., 2013. Implicit and explicit social mentalizing: dual processes driven by a shared neural network. Front. Hum. Neurosci. 7, 1–6. https://doi.org/10.3389/fnhum.2013.00560

Wake, S., Izuma, K., 2017. A common neural code for social and monetary rewards in the human striatum. Soc. Cogn. Affect. Neurosci. nsx092. https://doi.org/10.1093/scan/nsx092

Wolpert, D., Ghahramani, Z., Jordan, M., 1995. An internal model for sensorimotor integration. Science (80-. ). 269, 1880–1882. https://doi.org/10.1126/science.7569931

Wolpert, D.M., Doya, K., Kawato, M., 2003. A unifying computational framework for motor control and social interaction. Philos. Trans. R. Soc. London. Ser. B Biol. Sci. 358, 593–602. https://doi.org/10.1098/rstb.2002.1238

Yoshida, W., Seymour, B., Friston, K.J., Dolan, R.J., 2010. Neural mechanisms of belief inference during cooperative games. J. Neurosci. 30, 10744–10751. https://doi.org/10.1523/JNEUROSCI.5895-09.2010

Zaki, J., 2020. Integrating Empathy and Interpersonal Emotion Regulation. Annu. Rev. Psychol. 71, 517–540. https://doi.org/10.1146/annurev-psych-010419-050830

Zaki, J., 2014. Empathy: A motivated account. Psychol. Bull. 140, 1608–1647.

Zaki, J., Bolger, N., Ochsner, K., 2008. It Takes Two: The Interpersonal Nature of Empathic Accuracy. Assoc. Psychol. Sci. 19, 399–404. https://doi.org/10.5840/faithphil200219441

Zaki, J., Williams, W.C., 2013. Interpersonal emotion regulation. Emotion 13, 803–810. https://doi.org/10.1037/a0033839

Zhang, L., Lengersdorff, L., Mikus, N., Gläscher, J., Lamm, C., 2020. Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices. Soc. Cogn. Affect. Neurosci. 15, 695–707. https://doi.org/10.1093/scan/nsaa089

26