Chat-type Manzai Application: Mobile Daily Presentations based on Automatically Generated Manzai Scenarios Kazuki Haraguchi Kazuki Yane Akira Sato Konan University Konan University International Cancer Institute Kobe, Japan Kobe, Japan Osaka, Japan [email protected] Eiji Aramaki Isao Miyashiro Akiyo Nadamoto Nara Institute of Science and Osaka International Cancer Institute Konan University Technology Osaka, Japan Kobe, Japan Nara, Japan [email protected]

ABSTRACT Multimedia (MoMM ’20), November 30-December 2, 2020, Chiang Mai, Thai- We have proposed and demonstrate Manzai robots that automati- land. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3428690. 3429170 cally generate Manzai scenarios. Manzai is a Japanese traditional comedy consisting of two with funny dialogues like stand up comedy. Our Manzai robots are a huge system, but it is 1 INTRODUCTION not easy for users to watch Manzai every day using our Manzai We have proposed Manzai robots, which perform Manzai based on robots. As described herein, we propose a mobile style of presenting automatically generated Manzai dialogue based on AI techniques[7][1]. our automatically generated Manzai anytime and anywhere using Manzai, a Japanese traditional comedy, consists of two comedians a web application. We designate the application as a Chat-type with funny dialogue like stand up comedy. With our Manzai robot Manzai Application. The Chat-type Manzai Application is intended system, two robots perform the Manzai that we generate automati- to make people healthier by laughing as they relax and watch the cally. The generated Manzai scenario consists of funny dialogue for Manzai. However, the Chat-type Manzai Application loses direction. two robots. We have Manzai robots of two types: large and small. Therefore, we propose a new dialogue component: the Name list The large robot is about 1 m tall. The small robot is about 30 cm component. We used experiments of three types to measure the tall. Our Manzai robots consist of two robots, requiring much space benefits of our proposed application and component. to perform the Manzai. Robots are familiar to people. However, because of the Manzai robot size, watching Manzai is not always CCS CONCEPTS easy, especially for people who are hospitalized or receiving home • Information systems → Web applications; • Applied comput- care. ing → Health care information systems; • Human-centered com- By particularly addressing laughter, Nobori[10] proposes that puting → Human computer interaction (HCI). laughter affects both the respiratory physiology and psychiatric and neurological immunology. Patch Adams, a famous American doctor, KEYWORDS proposed HospitalClown, which provides laughter to patients to support care for their health. In this way, the laughter supports Web application, comedy, Manzai, mental health care, dialogue health care. However, people who provide laughter such as Hospi- genaration, scenario talClown are less. Therefore, we consider that we should provide ACM Reference Format: our Manzai robots anytime and anywhere. People can watch our Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, generated Manzai and becomes healthier. Nevertheless, our Manzai and Akiyo Nadamoto. 2020. Chat-type Manzai Application: Mobile Daily robots are huge. Therefore, is not easy for users to watch Manzai Comedy Presentations based on Automatically Generated Manzai Scenarios. anytime and anywhere. We infer that we should develop a system In The 18th International Conference on Advances in Mobile Computing and that allows easy watching of Manzai daily. Therefore, we specif- ically examined mobile devices such as smartphones and tablet terminals, which are pervasive in people’s daily life. For this study, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed we propose a web-based application system by which users can for profit or commercial advantage and that copies bear this notice and the full citation easily watch Manzai daily on a smartphone and tablet terminal. on the first page. Copyrights for components of this work owned by others than ACM Specifically, we propose a chat-type web application with balloons must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a of images of two characters using automatic generation of a Manzai fee. Request permissions from [email protected]. script that has been used with Manzai robots. We designate the MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand system as a “Chat-type Manzai Application". In this way, users © 2020 Association for Computing Machinery. ACM ISBN 978-1-4503-8924-2/20/11...$15.00 can watch generated Manzai easily and daily using the Chat-type https://doi.org/10.1145/3428690.3429170 Manzai Application. MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, and Akiyo Nadamoto.

In general, Manzai is important not only for the dia- performing comedy in real environments. They conducted robot logue but also the comedian direction such as movements. With -telling timing markedly improved the audience response. Kat- Manzai robots, we are directed by the robot movements. When we evas et al.[4] performed stand-up comedy use by humanoid robots. developed the Chat-type Manzai Application, we lost some aspects They demonstrated the manner in which humanoid robot can be of orientation. Therefore, we propose a new dialogue component useful to probe the complex social signals that contribute to the live to compensate for that lost aspect of orientation. performance experience. For this study, we specifically examined a As described herein, we contribute the following three points. more understandable and funny Manzai scenario, and assessed a • Proposing a new dialogue component for the Manzai sce- new style to watch an automatically generated Manzai scenario. nario. Numerous studies have assessed mental health care using robots. • Proposing a Chat-type Manzai Application. Shibata et al.[11] described robot therapy, which is mental health • Experiments conducted to measure benefits of the Chat- care using animal-type robots, using the seal-type mental com- type Manzai Application for fun and impressions based on mitment robot Palo, which aims to enrich daily life and heal peo- comparison with Manzai robots. ple’s minds. Wagemaker et al.[16] experimented with robot-based animal-assisted therapy. They show that it has no clear benefi- This paper is organized as follows: Section 2 presents discus- cial effect, but that positive interactions with the robot canbeof sion of related work. Section 3 includes a description of the Basic therapeutic value in itself. As described herein, we aim to care for concept of Manzai. Section 4 proposes a new dialogue component people’s minds by inducing emotions of laughter using robots and called the “Name list component”. Section 5 proposes the Chat-type applications to watch automatically generated Manzai scenarios. Manzai Application. Section 6 presents experiments. Finally, section 7 presents conclusions of this paper. 3 BASIC CONCEPT 2 RELATED WORK 3.1 What is Manzai? Numerous studies have presented dialogue generation. Chen et Manzai, a Japanese traditional comedy, usually poses a character al.[2] proposed Hierarchical Variational Memory Network called duo engaged in humorous dialogue. Manzai generally consists of HVMN, which uses a hierarchical structure and a variational mem- two roles: boke and tsukkomi. Boke states a misunderstanding, a ory network with a neural encoder–decoder network. Zhao et funny remark or a joke, which elicits laughter from the audience. al.[18] proposed pre-training based multiple knowledge syncretic Tsukkomi then points out mistakes and misunderstandings made transformer using one framework to integrate knowledge informa- by the boke and provides opportunities for humor. tion of multiple. Tuan et al.[14] proposed StepGAN, for which the In our system, when a user inputs keywords, the system creates discriminator is modified for automatic assignment scores, quan- a Manzai scenario based on web news related to a keyword. There tifying the goodness of a generated sub-sequence. Zhou et al.[19] are three parts of the act: the introduction, body, and conclusion. proposed the nCG-ESM system, which is sufficiently flexible to In this time, web news usually has bad news, such as incidents and allow extension to situations involving a broader range of emotions. accidents. The bad news should not be used for Manzai because Nie et al.[9] proposed a multimodal dialog system with adaptive de- Manzai is comedy. We set stop words such as "death" and "disaster" coders to generate general responses, knowledge-aware responses, to prevent generation of an inappropriate Manzai scenario. Such and multimodal responses dynamically. This study examines a dif- subjects are seldom the subject of humor of any kind. ferent approach by which we particularly examine Manzai. Numerous studies have presented Manzai. Tsutsumi [13] con- 3.1.1 Introduction part. The Introduction is an important part of ducted a conversation analysis of “boke-tsukkomi” in Manzai. Ad- Manzai. It consists of greetings and getting the first laugh. In the ditionally, he conducted experiments showing English speakers the conventional method, greetings are exchanged in keeping with the Manzai translated to English and showed that laughter was trans- season. However, this greeting is unable to play the role of the mitted to English speakers with the translated Manzai. Hayashi et Introduction part because the same greeting is made each time. al.[3] proposed a Robot Manzai system using a pair of robots as a As described herein, we propose the greeting of the day as a new passive social medium. They conducted a comparison between the Introduction part. For this Introduction part, we obtain information “passive medium” and “passive social medium”. They conducted a related to the anniversary of the day from the web and generate a comparison experiment between “Robot Manzai” and “Manzai” by dialogue using this information. a human in videos. Then they demonstrate the usefulness of “Robot Manzai” as entertainment. Mashimo et al.[6][7] and Aoki et al.[1] 3.1.2 Body part. The Body part, the main part of a Manzai scenario, proposed automatic generation of Manzai scenarios. consists of humorous dialogues. This part explains the obtained Numerous studies have presented and humor using robots. news articles and gets many laughs by the performance of vari- Klaus et al.[17] proposed a real-time adaptation of a robotic joke ous humorous dialogues. We designate these blocks of humorous teller based on human social signals: facial smiles and vocal laughs. dialogue are dialogue component. We proposed components of They implemented an entertainment robot shown to learn jokes four types: Word-mistake component, Nori-tsukkomi component, that were in accordance with the user’s preferences without ex- Exaggeration component, and Rival words gap component. Com- plicit feedback. Knight et al.[5] propose Robot Theater as a novel ponents are created for each sentence in a news article separated framework to develop and evaluate the interaction capabilities of by punctuation marks. The Body part is composed by combining embodied machines. Vilk et al.[15] built robotic stand-up comedian, them. Mobile Daily Comedy Presentations based on Automatically Generated Manzai Scenarios MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Table 1: Tags in Manzai scenario

Tag name Explanation line Lines/strings that robots speak face Robots facial expression information balloon Lines/stings to be displayed on the screen position Robots facing direction information

Figure 2: Name list Component

4 NAME LIST COMPONENT When we generate the same Manzai scenario as the Manzai robots, we are unable to generate good Manzai. The reason is that we lose the direction which represents characters’ actions. We must add additional value to the new scenario. Furthermore, the purpose of developing the Chat-type Manzai Application is to make it easy for users to watch Manzai every day. In other words, we must avoid generation of a similar script every day. We propose a new dialogue component called the Name list component. Figure 2 presents an example of the Name list component. In the Name list component, the boke states that he likes the attributes of a person who appears in the news. At this time, the attribute characterizes the person, such as person’s occupation. We deter- mine attributes from the first sentence of the Wikipedia article. We designate the original attribute. In answer, the Tsukkomi asks for the other person, who has the same attributes. Next, the boke states Figure 1: Example of our Manzai scenario the names of many people. However, half of them differ in attributes to the first person who appears in the news. Therefore, the boke says the misunderstood dialogue. Laughter occurs among users. We designate the misunderstanding attribute as a false attribute. In the example of Figure 2, the boke first says figure skaters, butin 3.1.3 Conclusion part. The Conclusion part includes a farewell the middle of the dialogue, he says gymnasts. The original attribute and a final laughing point. In our system, the Conclusion partisa is figure skating. The false attribute is gymnastics. riddle of words related to web news. The flow of name list component generation is explained below. (1) Extracting the person’s name who is written in Wikipedia 3.2 Format of Manzai scenario from the news. We consider that persons described in Wikipedia are almost The automatically generated Manzai scenario is in XML format all famous person.Then we use Wikipedia. (show Figure 1). The command tag consists of each speaker’s ac- (2) Extracting the person’s attribute (original attribute) from tion and line. The command tag includes a line tag, a face tag, a Wikipedia. balloon tag, and a position tag, which are necessary for the action. (3) Extracting names of people who have the original attribute. Descriptions of the respective tags are given as Table1. All tags (4) Generating a false attribute. have “target” attributes. They indicate the speaker of the dialogue, (5) Extracting names of people who have the false attribute. who is tsukkomi or boke. MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, and Akiyo Nadamoto.

Figure 3: Flow of generating Name list component

4.1 Extraction of false attribute When we extract false attributes, we consider that the distance from the original attribute is important. This attribute distance represents the ontological distance between original attribute and false attribute. The attribute distance is short. The user does not feel surprised and does not laugh. However, it is too long: the user does Figure 4: Chat-type Manzai Application not understand the dialogue and does not laugh. Then we consider the following two definitions to determine the attribute distance. Table 2: Facial expression icons Definition 1: Common hypernyms among attributes. We determine the distance of attribute using a tree struc- neutral anger angry disagreement ture of the Wikipedia category. In the tree structure of the Wikipedia category, the category with the same mother node of original attribute becomes a candidate of the false cate- happiness haughtiness bliss affection gory. Definition 2: Similar usage in sentences. One feature of Word2Vec[8] is "Words that are similarly used generate similar vectors". Therefore, we infer that the false at- pity pleasantness joy sadness tribute is used similarly to the original attribute. Then we use Word2Vec. As described herein, we use NWJC2Vec[12] from the National Institute for Japanese Language and Linguistics. fear shame surprise tension The degree of similarity between two words in Word2Vec is 0.7 or fewer is extracted.

4.2 Enumerate names We extract names of people who are the same original or false at- news articles, which are headline news on time. Dialogue balloons tributes from Wikipedia. In the category structure of Wikipedia, we are added to the HTML file as the Manzai progresses. The screen extract all names that have the same original/false attributes. They automatically scrolls to show it at the time. The dialogue balloons become candidates of enumerate names. When the recognition of are drawn with a standard feature of the style sheet. The line breaks the enumerate names is low, it is difficult for a user to understand depend on the number of characters in the dialogue and the width of and laugh with the dialogue. We measure the degree of recogni- the web browser. The Chat-type Manzai Application also uses facial tion of each name. Therefore, a threshold is set for the degree of expression icons of 16 types (show Figure 2) to express emotions. recognition of extracted names. The degree of user recognition is The Chat-type Manzai Application procedure is the following. reflected in the number of Google search results. When the degree (1) Automatic generation of a Manzai scenario. of recognition of the extracted name exceeds a certain threshold, it (2) Making a CSV file, which is an application scenario, from is extracted as a false attribute. original scenario XML files. The CSV file includes informa- tion of speakers, sentences of dialogues, and expression of 5 CHAT-TYPE MANZAI APPLICATION each character. We propose a Chat-type Manzai Application, which is a new style of (3) Making an audio file using speech synthesis. The audio file watching Manzai on mobile devices such as a smartphone or tablet, consists of a dialogue. providing opportunities to laugh in everyday life. Figure 4 presents (4) When the Application is launched and the play button is a display image of chat-type Manzai. Figure 4 shows the routine, pressed, the system accesses the CSV file in each dialogue. with the left side as tsukkomi and right side as boke. Each dialogue (5) When audio plays, the system simultaneously displays dia- is displayed by balloons and audio on a web browser. The Manzai logue balloons and facial expression icons corresponding to is generated automatically on the setting time. It generated by the speaker. Mobile Daily Comedy Presentations based on Automatically Generated Manzai Scenarios MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Figure 5: Procedure of the Chat-Type Manzai Application

(6) When audio is ended, the system loads the next audio file and plays it. (7) Repeat (5) and (6) until the last audio file. The server side computes (1)–(3). The client (smartphone or tablet) computes (4)–(7). Then (1)–(3) are executed automatically on time every day. Users can watch Manzai using the Chat-type Manzai Application. The user can play or stop the Manzai at any time by double-clicking or double-tapping the screen on the web. We develop the generated Manzai scenario using Python and develop Figure 6: Result of Experiment1 the interface using JavaScript. The operating environment is Google Chrome.

6 EXPERIMENTS (1) Can you understand which block is the dialogue component As described in this paper, we conduct three experiments to measure related to the name? benefits of our proposed system. (2) Is the dialogue component funny? (3) Is the Manzai easy to understand? 6.1 Experiment 1: Name list component (4) Is the Manzai funny? We conducted an experiment to ascertain whether the proposed Subjects answer question (1) to yes or no. Then they answer the name list component makes the produced text more easily and question (2)–(4) using a five-point Likert scale (5, suitable; 1, un- makes it more understandable than our conventional method. suitable). For each question, subjects must answer with reasons for Condition the evaluation. This study examined six men and women subjects in their 20s. Results and Discussion We compared our proposed method with the baseline method. The Figure 6 presents result of the experiment. As shown in Figure baseline is Manzai based on the conventional method. We use the 6(2), 40% of subjects answered that it was funny. The reason is that same query both for our proposed method and for the baseline. “It’s a good idea to change naturally into a false attribute”. How- The query is "Naomi Osaka": a famous tennis player. Name list ever, some subjects answered “I can’t understand this component component’s original attribute is tennis. The false attribute is golf. because I don’t know many false attribute”. We must consider the The enumerated names are Naomi Osaka, Serena Williams, Novak degree of recognition of the false attribute. From Figure 6(3), then Djokovic, Roger Federer, Ai Sugiyama, Masahiro Kuramoto, Yuuta by the comparing the baseline with the proposed method, we can Ikeda, Singo Katayama, Hideki Matsuyama, and Ryo Ishikawa. understand that the proposed method is more understandable than First, we present the baseline to subjects. After watching the base- the baseline. From Figure 6(4), in the proposed method, there are line Manzai, they answer the questions. Next, we present our pro- 50% of subjects answered that it was almost suitable. In contrast, in posed name list component to them and, after watching the name the baseline, no answer is both suitable and almost suitable. Results list component Manzai, they answer the same questions. We ask demonstrated that we can understand that the proposed method is the subject four questions: funnier than the baseline. MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, and Akiyo Nadamoto.

Table 3: Conditions used for experiments Hypothesis 4 The Chat-type Manzai Application is more in- teresting than the Manzai robots. #subject Media and Scenario #men #women We consider comparison of the evaluation results of the Chat- 5 type Manzai Application and the Manzai robot, as well as the eval- First App(Scenario1)→Robots(Scenario2) 3 2 uation results of the media for each subject. 5 Hypothesis 1: Manzai robots are funnier than the Chat-type Second Robots(Scenario1)→App(Scenario2) 2 3 Manzai Application. 5 Figure 7 and Figure 8 present results of question 1. As presented Third App(Scenario2)→Robots(Scenario1) 3 2 in Figure 7, in the Chat-type Manzai Application, the number of 5 subjects who answered that it was funny is almost equal to the Fourth Robots(Scenario2)→App(Scenario1) 2 3 number who answered that it was not funny. However, with the Manzai robots, more than half of the subjects answered that it was not funny. As shown in Figure 8, five subjects rated the Chat-type 6.2 Experiment 2: Media comparison Manzai Application as funnier than the Manzai robots. Only one subject answered that the Manzai robots funnier than the Chat- experiment type Manzai Application. Half of the subjects answered that both We conducted experiment 2 to measure the benefit of our proposed media are equally funny. The results demonstrated that the Chat- Chat-type Manzai Application by comparison with Manzai robots. type Manzai Application is rated as funnier than the Manzai robots. Condition Hypothesis 1 is rejected as false. The reason is that the funny aspect The subjects were 10 men and 10 women in their 20s, from whom of Manzai might be influenced by its ease of understanding. consent had been obtained. We conducted four experiments with Hypothesis 2: The Chat-type Manzai Application is more un- different subjects. There are five subjects per experiment. Table derstandable than the Manzai robots. The difference is large. 3 presents condition of experiments. The reason for dividing ex- The result of question 2 are shown in Figures 9 and 10. The re- periments is to avoid the difference of results depending on the sult of question 3 are shown in Figures 11 and 12. As shown in conditions. In the case of the first-term experiment, the subjects Figure 9 and Figure 11, for the Chat-type Manzai Application, 90% first watch a Chat-type Manzai Application (Figure 4) and answer of subjects responded that it was easy to understand; 80% subjects the questions. Next, they watch Manzai robots and then answer the responded that it was easy to understand Manzai components. As questions. These media’s Manzai scenarios are differ. shown in Figure 10, it is a viewpoint of understandable for each The following are the questions posed to subjects. medium, 14 of the subjects answered that the Chat-type Manzai (1) Is the Manzai on watching media(Manzai robots or Chat- Application was more understandable. As shown in Figure 12, it type Manzai Application) funny? is an understandable viewpoint for the dialogue component: 13 (2) Is the Manzai on watching media easy to understand? of the subjects answered that the Chat-type Manzai Application (3) Did you understand the Manzai component on watching was more understandable. Results showed that we can understand media? that the Chat-type Manzai Application is better than the Manzai (4) Did you have a realistic feeling in the Manzai on watching robots. We surmise that the reason is that the Manzai robot pro- media? vides dialogue only on audio. In contrast, the Chat-type Manzai (5) Would you like to see another manzai on watching media? Application provides them audio and text. In the conventional Man- zai scenario include Word-mistake and riddle. They are difficult The evaluation was made in five steps, which are “suitable” as5, to understand on audio. The Manzai robot provides only audio, “almost suitable” as 4, “about the same” as 3, “somewhat unsuitable” Manzai becomes difficult to understand and loses its humor value. as 2, and “unsuitable” as 1. By contrast, the Chat-type Manzai Application provides audio and Results and Discussion text. It makes Manzai easier to understand and makes it funny. The The characteristics of the proposed Chat-type Manzai Application results demonstrated that hypothesis 2 is true. are that it is easy to understand as the text of the dialogues, and Hypothesis 3: Manzai robots have a more realistic feeling that it is easy to watch in everyday life because users can watch than the Chat-type Manzai Application. Manzai on a smartphone or tablet. On the other hand, the charac- Figure 13 and Figure 14 present question 4 results. Figure 13 teristics of Manzai robots are that it can watch the realistic feeling shows that the result of Manzai robots are slightly better than those performance by robots moving according to speech. Based on these of the Chat-type Manzai Application. However, the results of both characteristics of each medium, we formulate and evaluate the media are low. Furthermore, we devote attention to each subject in following four hypotheses. Figure 14: 8 subjects responded that the Manzai robots are more Hypothesis 1 Manzai robots are funnier than the Chat-type realistic than the Chat-type Manzai Application; half of the subjects Manzai Application. answered that both media are similarly realistic. Before we had an Hypothesis 2 The Chat-type Manzai Application is more un- experiment, we considered that the Manzai robots are more realistic derstandable than the Manzai robots; the difference is large. than the Chat-type Manzai Application because the Manzai robots Hypothesis 3 The Manzai robots have a more realistic feeling have more direction which moves according to speech, and they than the Chat-type Manzai Application. are 3D robots. In contrast, the results of experiment showed the Mobile Daily Comedy Presentations based on Automatically Generated Manzai Scenarios MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Figure 7: (1)Is manzai funny? per media Figure 8: (1)Is manzai funny? per subject

Figure 9: (2)Is manzai easy to understand? per media Figure 10: (2)Is manzai easy to understand? per subject

Manzai robots are slightly more realistic than Chat-type Manzai As the results of experiment showed, the characteristics of the Application. Results showed that hypothesis 3 is true. However, Chat-type Manzai Application are easy to understand and easy it is necessary to evaluate how realistic feelings affect the fun of to watch in everyday life. In contrast, the characteristic of the the robot, and to ascertain whether the Manzai robot performance Manzai robots is that they can present realistic feeling during a needs a realistic feeling. performance because they can move according to speech. In this Hypothesis 4: The Chat-type Manzai Application is more in- way, each medium has its characteristics. The evaluations vary teresting than the Manzai robots. depending on subject interest. Therefore almost no difference was Figure 15 and Figure 16 present results of question 4. As Figure found between evaluation of both media. Additionally, little gender 15 shows, results of neither media are high. Furthermore, as Figure difference was found in responses to the questions. 16 shows, five subjects answered that the Chat-type Manzai Appli- cation is more interesting than the Manzai robots. In contrast, three subjects answered that the Manzai robots are more interesting than 6.3 Experiment 3: Everyday life experiments the Chat-type Manzai Application. Other subjects reported both using Face Scale media as about the same. The results showed that hypothesis 4 Experiment 3 was conducted to measure the Chat-type Manzai must be rejected as false. Application health effects when subject watched Manzai every day. We used the face scale to assess changes in a subject’s mood. MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, and Akiyo Nadamoto.

Figure 11: (3)Did you understand the manzai component? Figure 12: (3)Did you understand the manzai component? per media per subject

Figure 13: (4)Did you have a realistic feeling? per media Figure 14: (4)Did you have a realistic feeling? per subject

Condition on the day 5 scenario and day 6 scenario, the difference between The subjects examined for this study were 7 men and women before and after the Face Scale Score was less than in other scenar- in their 20s from whom consent had been obtained. They used the ios. Because the subjects’ feelings were little changed on day 5 and Chat-type Manzai Application every day during one week. Every because more than half improved on day 6, the Face Scale Score day, the subjects answered with the face scale before watching the was likely to have been influenced by the scenario. For this experi- Manzai, next they watched the Manzai. Finally, they answered the ment, we examine only one week. Based on those results, we infer face scale again. According to the Face Scale, subjects selected a that we should conduct more long-term experiments. Additionally, facial expression icon that best describes the person’s own feelings. correlation between the Face Scale Score and the Manzai scenario Figure 18 shows the face scale with the subjects’ selected facial indicate that it should probably be investigated. expression icons of five types. Results and Discussion Figure 17 presents results obtained for the daily face scale. About 7 CONCLUSION 33% of subjects (16 out of 49 times) showed improved Face Scale As described herein, we proposed a new style of watching our ratings after watching Manzai. Results shown for our proposed automatically generated manzai in everyday life using a web ap- Chat-type Manzai Application made the user feel a little brighter. plication. We designate the application as the Chat-type Manzai Table 4 presents details of the results. As shown in table 4, especially Application. The purpose of the Chat-type Manzai Application is to Mobile Daily Comedy Presentations based on Automatically Generated Manzai Scenarios MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Figure 15: (5)Would you like to see another manzai? per Figure 16: (5)Would you like to see another manzai? per media subject

Figure 17: Evaluation by Face Scale

Results obtained from comparison of the proposed Chat-type Man- zai Application with conventional Manzai robots show that the application is easier to understand and funnier than the robots. The Face Scale results indicate that about 33% of subjects feel changed: feeling a little brighter. That result indicates the usefulness of the proposed application. In the near future, we expect to conduct experiments in which Figure 18: Face Scale the proposed application can be used for longer periods to evaluate changes in feelings. make people more healthy by laughing while watching our Manzai. However, the Chat-type Manzai Application loses direction. There- fore, we propose a new dialogue component called the name list component. We conducted experiments of three types to measure ACKNOWLEDGMENTS the benefits of our proposed application and component. Theex- This work was partially supported by Research Institute of Ko- perimentally obtained results indicate that our proposed method nan University, and by JSPS KAKENHI Great Number 19H04218, can generate funnier a Manzai scenario that is easy to understand. 19H04221, and 20K12085. MoMM ’20, November 30-December 2, 2020, Chiang Mai, Thailand Kazuki Haraguchi, Kazuki Yane, Akira Sato, Eiji Aramaki, Isao Miyashiro, and Akiyo Nadamoto.

Table 4: Daily Result

day 1 day 2 day 3 day 4 before after difference before after difference before after difference before after difference Subject 1 4 4 0 4 3 -1 3 3 0 3 4 1 Subject 2 5 4 -1 4 4 0 3 4 1 3 4 1 Subject 3 3 4 1 4 4 0 3 3 0 2 3 1 Subject 4 5 4 -1 4 4 0 4 4 0 4 3 -1 Subject 5 3 3 0 4 4 0 2 4 2 5 3 -2 Subject 6 3 2 -1 1 4 3 3 3 0 1 1 0 Subject 7 3 2 -1 4 4 0 3 3 0 2 2 0 average -0.4 average 0.29 average 0.43 average 0 variance 0.53 variance 1.35 variance 0.53 variance 1.14

day 5 day 6 day 7 before after difference before after difference before after difference Subject 1 3 3 0 3 4 1 4 4 0 Subject 2 3 4 1 3 4 1 2 3 1 Subject 3 3 4 1 4 4 0 2 4 2 Subject 4 3 3 0 3 3 0 3 3 0 Subject 5 4 4 0 2 3 1 4 5 1 Subject 6 3 3 0 3 4 1 3 3 0 Subject 7 3 3 0 3 3 0 3 2 -1 average 0.29 average 0.57 average 0.43 variance 0.2 variance 0.24 variance 0.82

REFERENCES [11] Takanori Shibata and K. Wada. 2011. Robot Therapy: A New Approach for Mental [1] Satoshi Aoki, Tomohiro Umetani, Tatsuya Kitamura, and Akiyo Nadamoto. 2018. Healthcare of the Elderly - A Mini-Review. Gerontology 57 (06 2011), 378–386. Generating Manzai-Scenario Using Entity Mistake. In Advances in Network-Based https://doi.org/10.1159/000319015 Information Systems, Leonard Barolli, Tomoya Enokido, and Makoto Takizawa [12] Hiroyuki Shinnou, Masayuki Asahara, Kanako Komiya, and Minoru Sasaki. (Eds.). Springer International Publishing, Cham, 1007–1017. 2017. nwjc2vec: Word Embedding Data Constructed from NINJAL Web Japan- [2] Hongshen Chen, Zhaochun Ren, Jiliang Tang, Yihong Eric Zhao, and Dawei Yin. ese Corpus. Journal of Natural Language Processing 24, 5 (2017), 705–720. 2018. Hierarchical Variational Memory Network for Dialogue Generation. In https://doi.org/10.5715/jnlp.24.705 Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). [13] Hideo Tsutsumi. 2011. Conversation Analysis of Boke-tsukkomi Exchange in International World Wide Web Conferences Steering Committee, Republic and Japanese Comedy. New Voices 5 (12 2011), 147–173. https://doi.org/10.21159/nv. Canton of Geneva, CHE, 1653–1662. https://doi.org/10.1145/3178876.3186077 05.07 [3] K. Hayashi, T. Kanda, T. Miyashita, H. Ishiguro, and N. Hagita. 2005. Robot Manzai [14] Yi-Lin Tuan and Hung-Yi Lee. 2019. Improving Conditional Sequence Generative - robots’ conversation as a passive social medium. In 5th IEEE-RAS International Adversarial Networks by Stepwise Evaluation. IEEE/ACM Trans. Audio, Speech Conference on Humanoid Robots, 2005. 456–462. https://doi.org/10.1109/ICHR. and Lang. Proc. 27, 4 (April 2019), 788–798. https://doi.org/10.1109/TASLP.2019. 2005.1573609 2896437 [4] Kleomenis Katevas, Patrick Healey, and Matthew Harris. 2015. Robot Comedy [15] John Vilk and Naomi T. Fitter. 2020. Comedians in Cafes Getting Data: Evaluating Lab: Experimenting with the Social Dynamics of Live Performance. Frontiers in Timing and Adaptivity in Real-World Robot Comedy Performance. In Proceed- Psychology 6 (08 2015). https://doi.org/10.3389/fpsyg.2015.01253 ings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction [5] Heather Knight, Scott Satkin, Varun Ramakrishna, and Santosh Divvala. 2011. (Cambridge, United Kingdom) (HRI ’20). Association for Computing Machinery, A savvy robot standup comic: Online learning through audience tracking. In New York, NY, USA, 223–231. https://doi.org/10.1145/3319502.3374780 Workshop paper (TEI’10). [16] Eline Wagemaker, Tycho J. Dekkers, Joost A. Agelink van Rentergem, Karin M. [6] Ryo Mashimo, Tomohiro Umetani, Tasuya Kitamura, and Akiyo Nadamoto. 2015. Volkers, and Hilde M. Huizenga. 2017. Advances in Mental Health Care: Five N = Automatic generation of Japanese traditional funny scenario from web content 1 Studies on the Effects of the Robot Seal Paro in Adults With Severe Intellectual based on web intelligence. In In proceedings of the 17th International Conference Disabilities. Journal of Mental Health Research in Intellectual Disabilities 10, 4 on Information Integration Web-based Applications & Services. 173–165. (2017), 309–320. https://doi.org/10.1080/19315864.2017.1320601 [7] Ryo Mashimo, Tomohiro Umetani, Tasuya Kitamura, and Akiyo Nadamoto. 2015. [17] Klaus Weber, Hannes Ritschel, Florian Lingenfelser, and Elisabeth André. 2018. Human-Robots implicit communication based on dialogue between robots using Real-Time Adaptation of a Robotic Joke Teller Based on Human Social Signals. automatic generation of funny scenarios from web. In In proceedings of the In Proceedings of the 17th International Conference on Autonomous Agents and eleventh ACM/IEEE International Conference on Human Robot Interaction. 327– MultiAgent Systems (Stockholm, Sweden) (AAMAS ’18). International Foundation 334. for Autonomous Agents and Multiagent Systems, Richland, SC, 2259–2261. [8] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. [18] Xiangyu Zhao, Longbiao Wang, Ruifang He, Ting Yang, Jinxin Chang, and Ruifang Distributed Representations of Words and Phrases and their Compositionality. Wang. 2020. Multiple Knowledge Syncretic Transformer for Natural Dialogue In NIPS. Curran Associates, Inc., 3111–3119. Generation. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW [9] Liqiang Nie, Wenjie Wang, Richang Hong, Meng Wang, and Qi Tian. 2019. Mul- ’20). Association for Computing Machinery, New York, NY, USA, 752–762. https: timodal Dialog System: Generating Responses via Adaptive Decoders. In Pro- //doi.org/10.1145/3366423.3380156 ceedings of the 27th ACM International Conference on Multimedia (Nice, France) [19] Guangyou Zhou, Yizhen Fang, Yehong Peng, and Jiaheng Lu. 2019. Neural (MM ’19). Association for Computing Machinery, New York, NY, USA, 1098–1106. Conversation Generation with Auxiliary Emotional Supervised Models. ACM https://doi.org/10.1145/3343031.3350923 Trans. Asian Low-Resour. Lang. Inf. Process. 19, 2, Article 19 (Sept. 2019), 17 pages. [10] Mikio Nobori. 1994. Discussion of laughter for Medical viewpoints(in Japanese). https://doi.org/10.1145/3344788 In The research of laghter society 1. 26–30.