A Chatbot with Interpersonal Communication Recognition: Determine the Position on Leary’S Rose After
Total Page:16
File Type:pdf, Size:1020Kb
Running head: CHATBOT WITH INTERPERSONAL COMMUNICATION RECOGNITION 1 A chatbot with interpersonal communication recognition: determine the position on Leary’s Rose after automatic text analyzation. Name: Gerard Johan Visser Student number: S1068008 Date: 20-08-2018 Supervisor: Dr. P. Haazebroek Second reader: Dr. R. E. de Kleijn CHATBOT WITH INTERPERSONAL COMMUNICATION RECOGNITION 2 Abstract The research area of chatbots is relatively young and therefore the goal of this study is to gather more information about text categorization by chatbots. At this time chatbots are increasingly used in online communication with users. It is a challenge to let chatbots respond appropriately on an emotional level in a way that users experience the answers as a positive interaction. The current study examined the possibility of mapping this interaction using text analysis with the LIWC and classification on Leary’s Rose. Based on Leary’s Rose we predicted the positive experience with NPS. This study consists of three phases. The first phase is a text analysis to scale sentences on Leary’s Rose. The sentences were scaled by 102 participants on two scales (the “I & We” scale and the “Dominance & Submissive” scale). With these scaled sentences, a classifier (classifier A) is created and trained with the LIWC and a regression analysis. The results of phase one suggests that our database contains mostly “Dominance/I” and “Submissive/We” sentences. Classifier A (80.8%) is 3% better than the random baseline (77.8%). Classifier A is tested in phase two with self-annotated sentences. These self-annotated sentences were from 15 participants on two scenarios’. Based on these self-annotated sentences we created also two new classifiers (B1 & B2). The test results of classifier A (59.0%) is the same as the (59.0%) random baseline. The two new classifiers (B1 & B2) created in phase two performed better (B1: 72.3% and B2: 76.6%) than the random baseline (59.1%). In phase three we tried to predict NPS on Leary’s rose. Based on Kendalls tau-b correlation and crosstabs we compared the classifiers. The findings suggested that it is possible to predict NPS based on Leary’s Rose. A possible implication is the need of a multimodal approach for text analysis. Future research should focus on better ways of annotation to prevent skewed, small and noisy databases. More implications and suggestions are presented in the discussion. Keywords: emotion detection, emotion classification, text analysis, Leary’s Rose, interpersonal communication, chatbot, Net Promoter Score (NPS) CHATBOT WITH INTERPERSONAL COMMUNICATION RECOGNITION 3 A chatbot with interpersonal communication recognition: determine the position on Leary’s Rose after automatic text analyzation. In recent years, more and more chatbots became available in different areas. Via chatbot-software a human is able to interact with a computer in natural language. This software can extend daily life, for example a helpdesk chatbot (Rahman, 2012; Shawar, Atwell & Roberts, 2005) who is able to answer questions from customers. A customer would like to have information about a problem with a product and asks the chatbot. The chatbot is able to answer the question. But chatbots are also used in areas such as in educational tools (Keshtkar, Burkett, Li & Graesser, 2014; Vaassen & Daelemans, 2010) or in e-commerce and business (Chattaraman, Kwon & Gilbert, 2012). Because chatbots are used more and more, improvements should be made and one of the challenges is to detect emotion from the user in a chatbot conversation. To return to our ‘helpdesk chatbot’ example; imagine a helpdesk chatbot which can detect someone’s emotion. The chatbot is then able to change its type of communication based on the emotion of the user in a way he or she feels more understood. Thereby, the chatbot is able to recognize when the conversation goes sideways and transfer the customer towards a real human being. To reach this goal more research is needed. This study focuses on emotion classification of customer conversations with a helpdesk chatbot from a large online retailer. Chatbots A chatbot is a computer program designed to communicate with human users via natural language. The chatbot can recognize words or a group of words and based on this data the chatbot gives answers. This type of chatbot has certain benefits. First of all, a chatbot is always present and takes care for real-time events 24 hours a day. In addition, the communication by a chatbot is a dialog and is more effective than a monolog, when dealing with humans (Tatai, Csordás, Kiss, Szaló & Laufer, 2003). Furthermore, the chatbot combines large amounts of information and only shows the information that is asked for by the user. And at last, a chatbot can handle many cases simultaneously, which is cost-saving for the company because fewer employees are needed to answer the questions of the users. However, at this moment, there are challenges to deal with due to the complexity of human language and emotions. Computers have difficulties in understanding the endless variability of expression in how words are meant in language use to communicate meaning (Hill, Ford & Farreras, 2015). To create a computer program that is capable to interact with a person at a CHATBOT WITH INTERPERSONAL COMMUNICATION RECOGNITION 4 human level, requires the machine to understand human behaviors. One of the most important things in a conversation is expressing and understanding emotions and affects (Picard & Picard, 1997; Salovey & Mayer, 1990). The possibility to test humanized machines was proposed by Alan Turing in his “Turing Test” (Turing, 1950). This test is based on a conversation between a computer and a human judge. It is based on the ability of a computer program to impersonate a human, with the judge not being able to distinguish between a computer of a human being. One of the first chatbots subjected to the Turing Test was ELIZA, which is created at the Massachusetts Institute of Technology by Weizenbaum. ELIZA is a chatbot that emulates a psychotherapist (Weizenbaum, 1966). After ELIZA, a lot of other chatbots are created with different purposes. Still none of the chatbots passed the Turing Test (Saygin, Cicekli & Akman, 2000; Warwick & Shah, 2016). Emotion and classifying emotions As described above, emotion is an important factor in humanizing computers. Classifying customers emotions is important for companies because emotion has an effect on customer loyalty and satisfaction (DeWitt, Nguyen & Marshall, 2008; Varela-Neira, Vázquez-Casielles & Iglesias-Argüelles, 2008; Yu, White & Xu, 2007). To measure customer satisfaction and loyalty a company can use the Net Promoter Score (NPS). Shaw (2016) introduced a new indicator for emotional value, the Net Emotion Value (NEV). The NEV measures the emotion value towards a company. His work shows that the higher the NEV (positive emotion), the higher the NPS and thus has emotion an effect on the NPS. To extract emotion from text in a chat environment is different than extracting emotion from face to face interactions between humans. Emotion extraction from text is missing facial expressions, intonation of voice and body language (Vaassen, 2014). This complicates the emotion extraction from text. Another difference is the difference in communication styles in a human-chatbot versus a human-human chat-conversation. Users tend to be more agreeable, open, extrovert, conscientious, and self-disclosing when interacting with a human. When a human consciously interacts with a chatbot they report lower perceived attractiveness, less goal driven and more brutal language than in chats with humans (Mou & Xu, 2017). Hill and others (2015) found differences between human-human and human-chatbot communication in: more messages, shorter message lengths, more limited CHATBOT WITH INTERPERSONAL COMMUNICATION RECOGNITION 5 vocabulary and greater use of profanity. These differences in interaction should be taken into account when classifying emotion from chatbot texts. Interpersonal interaction To achieve emotion recognition in a chat conversation a real time automatic emotion analysis is needed. In 2010, Vaassen and Daelemans introduced the automatic classification of text according to a framework for interpersonal communication (Vaassen & Daelemans, 2010; Vaassen & Daelemans, 2011; Vaassen, Wauters, van Broeckhoven, van Overveldt, Daelemans & Eneman, 2012; Vaassen, 2014). This approach focuses not only on emotion classification but also the interaction between a human and a chatbot, which is very helpful. Several different frameworks for interpersonal communication have been developed over the past years (Gurtman, 2009). The first interpersonal communication model was created by the Kaiser Research Group by the name interpersonal circle, better known as “Leary’s Rose” (Leary, 1957). This framework determines two roles, a speaker and a listener. These two roles change during the conversation, when someone speaks, he is the speaker and when someone listens, he is the listener. The graphical representation of Leary’s Rose is a circle which is split vertically in the “I” and “We” side and horizontally in the “Dominance” and “Submission” side. The horizontal axis determines if the speaker is dominant or submissive towards the listener. The vertical axis determines the speaker’s willingness to co- operate. These partitions create four quadrants: “Lead”, “Follow”, “Defend” and “Attack.” Each quadrant can again be divided into two octants, which create in total eight