Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence The Influence of Emotion Expression on Perceptions of Trustworthiness in Negotiation Dimitrios Antos1, Celso De Melo2, Jonathan Gratch2 and Barbara Grosz1 1Harvard University, Cambridge, MA 02138, USA 2Institute for Creative Technologies, University of Southern California, Los Angeles, CA 90094, USA {dantos, grosz}@eecs.harvard.edu, [email protected], [email protected] Abstract People negotiate over how to schedule activities when par- ticipants have different time constraints and priorities, to When interacting with computer agents, people make inferences about various characteristics of these agents, efficiently allocate valuable resources across individuals or such as their reliability and trustworthiness. These per- corporations with varying needs and preferences, or to re- ceptions are significant, as they influence people’s be- solve international conflicts without resorting to violence. havior towards the agents, and may foster or inhibit re- The significance of negotiation has led to a large literature peated interactions between them. In this paper we in- in the fields of psychology, economics, sociology and com- vestigate whether computer agents can use the expres- puter science (e.g., see (Lewicki, Barry, and Saunders 2010), sion of emotion to influence human perceptions of trust- (Raiffa 1985) and (Jennings et al. 2001)). Computer agents worthiness. In particular, we study human-computer in- have been used in many negotiation settings, sometimes act- teractions within the context of a negotiation game, in ing on behalf of humans (Jennings et al. 2001) and some- which players make alternating offers to decide on how times negotiating with them (Lin and Kraus 2010). As the to divide a set of resources. A series of negotiation games between a human and several agents is then fol- scale and complexity of the domains in which negotiation lowed by a “trust game.” In this game people have to is employed are expected to increase, we foresee a growth choose one among several agents to interact with, as in the use of computer agents as negotiators; examples of well as how much of their resources they will trust to such domains might include traffic management (Kamar and it. Our results indicate that, among those agents that Horvitz 2009) and commerce (Maes, Guttman, and Moukas displayed emotion, those whose expression was in ac- 1999), among others. Furthermore, computer agents have cord with their actions (strategy) during the negotiation shown potential (compared with human negotiations) for game were generally preferred as partners in the trust improving negotiation outcomes in some cases (Lin, Oshrat, game over those whose emotion expressions and ac- and Kraus 2009). tions did not mesh. Moreover, we observed that when emotion does not carry useful new information, it fails Yet negotiation studies with computer agents have largely to strongly influence human decision-making behavior overlooked the fact that humans use significant verbal and in a negotiation setting. non-verbal cues when they negotiate (Drolet and Mor- ris 2000). The expression of emotion, in particular, has been shown to significantly influence negotiation outcomes Introduction (Barry, Fulmer, and Goates 2006; Van Kleef, De Dreu, It has been well-documented that humans treat computers and Manstead 2010). For instance, displaying anger was as social agents, in that they perceive in them human traits, shown to be effective in forcing larger concessions out of expect socially intelligent responses, and act toward them the other party, whereas positive emotion was found to be in a socially appropriate manner (Nass and Moon 2000; helpful in exploring and achieving mutually beneficial (inte- Nass 2004). This leaves open the possibility that an agent grative) solutions (Van Kleef, De Dreu, and Manstead 2004; may influence how humans perceive it through its presence Carnevale and Pruitt 1992). The effects of emotion during and behavior. The agent might find it advantageous to do business negotiations were also shown to be modulated by so if such perceptions stand to influence the interaction out- culture (Leung et al. 2005). In most studies in which emo- come, or perhaps the likelihood that similar interactions will tion was expressed by computer agents, this emotion was take place in the future. In this paper we focus on human conveyed by means of text sent by the computer agent to its perceptions of trustworthiness, and study the extend that human partner (e.g., in (Van Kleef, De Dreu, and Manstead these may be influenced by agent expressions of emotion in 2004) the computer would say “this offer makes me re- a computer-human negotiation environment. ally angry”). More recent implementations of virtual agents Negotiation is a commonly-used method for parties with tested for differences among different modalities of emotion diverging interests to reach a mutually-beneficial agreement. expression (e.g., text or virtual face) and found no significant Copyright c 2011, Association for the Advancement of Artificial differences (de Melo, Carnevale, and Gratch 2010). Intelligence (www.aaai.org). All rights reserved. We make two contributions in this paper. First, we inves- 772 tigate the effects of emotion on people’s perceptions of an The game proceeds by means of alternating offers, and agent’s trustworthiness. Such perceptions are formed in the participants play in turns, with the human always making course of human-computer interactions (in our case, nego- the first offer in a game. An offer by player i ∈{h, a} tiations; see (Bos et al. 2002)). We obtain robust results in- consists of a complete allocation of all coins between the ci (t) dicating that the expression of emotion does influence per- two participants. We use the notation j to denote how ceptions of trustworthiness. In particular, an agent is being many items of type j ∈{1, 2, 3, 4} player i was given in perceived as more trustworthy when its expressed emotion the offer made at round t ∈{1, 2,...}. Hence, allowing is in accord with its actions during the interaction. only for complete allocations means that in every round t, ch(t)+ca(t)=3, ∀j Second, we look at the effect of expressed emotion on offers must satisfy j j . In every round negotiation outcomes. Our results reveal only small influ- t>1 the negotiator whose turn it is may accept an offer, in ences on most negotiation metrics, in contrast with many which case the game ends and both participants make their well-documented findings (e.g., anger resulting in higher corresponding points; for the human player, these would be πh = whch(t − 1) concessions by the human). We hypothesize that this is a j j j . Alternatively, she may reject the of- consequence of the fact that the negotiation strategies of fer, and counter-offer a different split of the items. Finally, our agents were too “scripted” and predictable. As a result, at any point in the game, either participant may drop out.A the expressed emotion did not convey new information that game consists of a maximum of 15 rounds.2 If no agreement could be used by humans in their decision-making. With- is reached in any of these rounds, or if either player drops out the emotion carrying an important informational signal, out, both players make zero points. people seem to ignore it. The agents in our experiment differed in two ways: with These observations show that the expression of emo- respect to their strategy, and with respect to their emotion ex- tion holds potential benefits in human-computer decision- pression. The strategy of an agent encompasses when offers making, and should be viewed as a key part of agent design. are accepted or rejected, and what counter-offers are made However, deploying emotion successfully is not a simple by it. Likewise, the emotion expression of an agent defines matter, because the appropriate expression is very much de- whether emotion is expressed, and what type of emotion is pendent on context. Moreover, our results suggest that emo- displayed in each circumstance. Below we discuss the strate- tion can influence human decision-making behavior only gies and emotion expression policies we used in our agents; when it has the ability to convey new information beyond we also explain how emotion was displayed to the human. the observed decisions of the agent. Strategies of computer agents Experiment Design The strategy of an agent prescribes how the agent behaves To investigate the effect of emotion in human-computer ne- as a negotiator. Although the literature on effective negotia- gotiation and perceptions of trustworthiness we developed tion strategies is extensive (Sycara and Dai 2010), we lim- the following game: A human subject (h) is paired with a ited ourselves to simple, intuitive strategies for this exper- computer agent (a), and they must negotiate on how to di- iment. Our goal was not to exhaustively explore the effect vide a set of resources amongst themselves. The resources of emotion expression given complex strategies, but to as- consist of virtual “coins” of four types: gold, silver, bronze sess whether emotion has any effect on people’s behavior in and iron. In each game, there are three (3) coins of each type. computer-human negotiation, and whether this effect is de- Before the game starts, people are told the relative value of pendent upon the agent’s strategy. each coin type, i.e., that gold coins are more valuable than The strategies we used varied along two dimensions: silver, which are more valuable than bronze, which are more “flexibility” and “self-interestedness.” An agent follows a valuable than iron coins. However, people are not given the flexible strategy if its offers change from round to round exact numeric value (in points) of each coin type.1 Subjects throughout the game; its strategy is inflexible if it always are also informed that the relative valuation of the items by makes the same offer (or very similar ones) in every round.