Automatic Affective Feedback in an Email Browser
Total Page:16
File Type:pdf, Size:1020Kb
Automatic Affective Feedback in an Email Browser Hugo Liu Henry Lieberman Ted Selker Software Agents Group Software Agents Group Context-Aware Computing Group MIT Media Laboratory MIT Media Laboratory MIT Media Laboratory Cambridge, MA 02139 Cambridge, MA 02139 Cambridge, MA 02139 +1 617 253 5334 +1 617 253 0315 +1 617 253 6968 [email protected] [email protected] [email protected] ABSTRACT delighted us, the text sits unmoved in cold, square boxes on This paper demonstrates a new approach to recognizing and the computer screen. Nass et al.’s study of human- presenting the affect of text. The approach starts with a computer social interaction reveals that people naturally corpus of 400,000 responses to questions about everyday expect their interactions with computers to be social and life in Open Mind Common Sense. This so-called affective, just as with other people! [20],[21]. commonsense knowledge is the basis of a textual affect Sadly though, people have been so conditioned to expect so sensing engine. The engine dynamically analyzes a user’s little from the user interfaces of today that we are not even text and senses broad affective qualities of the story at the bothered by their inability to affectively respond to us like a sentence level. This paper shows how a commonsense friend or family member might do. affect model was constructed and incorporated into Chernov face style feedback in an affectively responsive This shortcoming in current user interfaces hinders email browser called EmpathyBuddy. This experimental progress in the bigger picture too. If software is to system reacts to sentences as they are typed. It is robust transform successfully into intelligent software agents, a enough that it is being used to send email. The response of social-affective connection between the user and computer the few dozen people that have typed into it is dramatically must be established because the capacity for affective enthusiastic. interaction plays a vital role in making agents believable [2],[27]. Without it, it will be hard to build trust and This paper debuts a new style of user interface technique credibility in the human-computer relationship. for creating intelligent responses. Instead of relying on specialized handcrafted knowledge bases this approach All of this gives rise to the question: Can a user interface relies on a generic commonsense repository. Instead of react affectively with useful and believable responsiveness relying on linguistic or statistical analysis alone to to a user engaged in a storytelling task like email or “understand” the affect of text, it relies on a small society weblogging? We argue that the answer is yes! In this of approaches based on the commonsense repository. paper, we present a commonsense-based textual analysis Keywords technology for sensing the broad affective qualities of everyday stories, told line-by-line. We then demonstrate Emotion and Affective UI, Agents and Intelligent Systems, this technology in an affectively responsive email browser Context -Aware Computing, User and Cognitive models. called EmpathyBuddy. EmpathyBuddy gives the user INTRODUCTION automatic affective feedback by putting on different One of the impressive triumphs of the computer revolution Chernov-style emotion faces to match the affective context is that it has given us more effective tools for personal and of the story being told through the user’s email. We social expression. Through emails, weblogs, instant evaluate the impact of the system’s interactive affective messages, and web pages, we are able to share our response on the user, and on the user’s perception of the experiences with friends, family, co-workers, or anyone system. else in the world who will listen. We use these mediums Paper’s Organization on a daily basis to share stories about our daily lives. This paper is structured as follows: First, we put our However, as useful as these tools have become, they still approach into perspective discussing existing approaches to lack the highly treasured social interactivity of an in-person textual affect sensing and other related work. Second we conversation. Much as we desire to relate stories of motivate our commonsense treatment of emotions with experiences that have saddened, angered, frustrated, and research from the cognitive psychology and artificial intelligence literature. Third, we discuss methods for constructing and applying an commonsense affect model. LEAVE BLANK THE LAST 2.5 cm (1”) OF THE LEFT Fourth, we discuss how a textual affect sensing engine was COLUMN ON THE FIRST PAGE FOR THE incorporated into Chernov face style feedback in an COPYRIGHT NOTICE. affectively responsive email browser called EmpathyBuddy, and we examine a user scenario for our system. Sixth, we present the results of a user evaluation of sufficiently large text input. So while these methods may EmpathyBuddy. The paper concludes with a summary of be able to affectively classify the user’s text on the page or contributions, and plans for further research. paragraph-level, they will not work on smaller text units THE APPROACH IN PERSPECTIVE such as sentences. Affective behavior is not only an important part of human- While page or paragraph-level sensing has its applications, human social communication [20], but researchers like there will be many applications for which this is not Picard have also recognized its potential and importance to granular enough. Synthetic agents may demand the higher human-computer social interaction. [25], [21]. In order for level of interactivity that can only be met by sensing the computers to make use of user affect, the user’s affective affect of individual sentences. [17],[2],[10]. Affective state must invariably first be recognized or sensed. speech synthesis will benefit from affect annotations of text Researchers have tried detecting the user’s affective state in at the sentence-level. [5]. Some context -aware systems will many ways, such as, inter alia, through facial expressions want to be able to react to the affective state of the user as [20],[1], speech [11], physiological phenomena [29], and captured in a single sentence or command. [16]. text [3],[10]. This paper addresses textual affect sensing. A Commonsense Knowledge Based Approach In the following subsections, we review existing Our proposed approach uses a large-scale (on the order of approaches to textual sensing and compare these to our ½ million facts) knowledge base filled with commonsense approach. about the everyday world, including affective Existing Approaches commonsense, to construct the user’s “commonsense affect Existing approaches to textual affect sensing generally fall model.” This model is premised on the observation that into one of two categories: keyword spotting, and people within the same population tend to have somewhat statistical modeling. similar affective attitudes toward everyday situations like Keyword spotting for automatic textual affect sensing is a getting into a car accident, have a baby, falling in love, very popular approach and many of these keyword models having a lot of work, etc. One likely explanation for this is of affect have gotten quite elaborate. Elliott’s Affective that these attitudes are part of our commonsense knowledge Reasoner [10], for example, watches for 198 affect and intuition about the world, which is shared across people keywords (e.g. distressed, enraged), plus affect intensity within a cultural population. modifiers (e.g. extremely, somewhat, mildly), plus a A textual affect sensing engine uses the constructed handful of cue phrases (e.g. “did that”, “wanted to”). commonsense affect model to try to sense broad affective Ortony’s Affective Lexicon [23] provides an often-used qualities of the text at the sentence level. We believe that source of affect words grouped into affective categories. this approach addresses many of the limitations of the Even with all its popularity, keyword spotting is not very existing approaches. robust in practice because it is sensing aspects of the prose Whereas keyword spotting senses only affective keywords rather than of the semantic content of a text. Affect in the prose, commonsense knowledge lets us reason about vocabulary may vary greatly from text to text, or may be the affective implications of the underlying semantic absent altogether. For example, the text: “My husband just content. For example, while the affect-keyword-based filed for divorce and he wants to take custody of my approach might work for the sentence “I was badly injured children away from me,” certainly evokes strong emotions, in a scary car accident,” only the commonsense knowledge- but lack affect keywords. A lot of affective communication based approach would work when the affect words are in a text is done without explicit emotion words, and in removed: “I was injured in a car accident.” these cases, keyword spotting would inevitably fail. Whereas semantically weaker statistical methods require Statistical methods can sometimes do better. By feeding a larger inputs, semantically stronger commonsense machine learning algorithm a large training corpus of knowledge can sense emotions on the sentence-level, and affectively annotated texts, it is possible for the system to thereby enable many interesting applications in synthetic not only learn the affective valence of affect keywords as in agents, affective speech synthesis, and context -aware the previous approach, but such a system can also take into systems mentioned above. account