2020 International Conference on Computational Science and Computational Intelligence (CSCI)

Explanation Generation in a Kabuki Dance Stage Performing Structure Simulation System

Miku Kawai Jumpei Ono Takashi Ogata Faculty of Software and Information Science Faculty of Software and Information Technology Faculty of Software and Information Science Iwate Prefectural University Aomori University Iwate Prefectural University Takizawa, Iwate, Japan Edogawa City, Tokyo, Japan Takizawa, Iwate, Japan [email protected] [email protected] [email protected]

Abstract—We are developing a stage performing structure The information collected was divided into two types: general simulation system. This paper introduces the generation of information and unique information on Kyōganoko Musume explanations into the system. First, we classified the explanations Dōjōji. For example, the term “Dōjōji” requires a general we collected. Thereafter, we prototyped a mechanism wherein the explanation of the building called Dōjōji, as well as an system automatically determines the content and method of an explanation of “Dōjōji” in the story. We obtained a general explanation based on arbitrary parameters. Through this trial, we explanation from Wikipedia and created a unique explanation will examine effective explanation methods for story generation. using [9]. Keywords— Dōjōji, Explanation, Stage Performing Structure, We used three types of explanations: general explanations, unique explanations, and unchiku explanations. The system has I. INTRODUCTION interest information, frequency information, and knowledge In an existing paper [1], the explanation of technical terms amount information. Interest information indicates how has been discussed. Another study has explained the problem- interested a user is in an item. Frequency information indicates solving process of AI systems [2]. This paper deals with the how much the description of the item was described at a given explanation of a story generation system. The explanations used point in time in the story. Knowledge amount information in this story support the understanding of users based on the indicates how much knowledge a user has about an item. For interests and knowledge of the users. interest information and knowledge amount information, 0 is the minimum value, and 10 is the maximum value. The insertion of This paper extends the method of expression of an an explanation is divided into two stages. The explanation explanation generation mechanism that we have developed. generation mechanism determines (A) the content of the Thus far, an explanation generation and an animation system explanation and (B) the method of explanation. have been developed. Both systems are based on the theme of kabuki (Kyōganoko Musume Dōjōji [3]). Kyōganoko Musume (A) Determining the content of the explanation is a process Dōjōji is a famous kabuki-dance work. Based on the Legend of for determining what to explain in a given situation. The system Anchin and Kiyohime, it has undergone many transformations, evaluates the candidate’s explanations based on frequency such as narratives and Noh, leading to the current kabuki-dance. information and interest information and determines what to Therefore, we have analyzed the stage performance structure of explain. Equation (1) is an evaluation equation for the Kyōganoko Musume Dōjōji in detail and developed an explanatory candidates. animation system that can depict the stage [4] [5]. The system 퐼 퐸 = (1) does not reproduce the details of the movements and swings of 퐹 the performer. Based on the story, the system generates code that describes character animation, background switching, and music E is the evaluation result, I is the interest information, and F playback timing. We consider explanations to be one of the is the frequency information (however, one is added to prevent rhetorical techniques of narration [6] [7]. The narrative division by zero). The system appreciates items that are of great generation system that we have developed includes an interest and underestimates items that have been explained many explanation generation mechanism. We have also implemented times. The system selects items for which the value of E is over-explained unchiku generation as one method for achieving higher than the threshold value (0.7). a rhetorical explanation. The aim of this paper is to propose an (B) Determination of the explanation method is a process for experimental explanation mechanism based on the interest and determining the procedure for explaining a specific explanation knowledge of users to consider the future impact of the item. The system determines the type of information to be explanation in the story generation system. explained (general, unique, or unchiku) based on the information of interest. The system determines that items with high interest II. DETAILED LOOK AT THE PROPOSED SYSTEM information should be explained more deeply, hence, it explains This paper examines how explanations can be inserted into unique information more extensively. However, the system animations. In the process of analyzing Kyōganoko Musume explains the storage information for items regarding which the Dōjōji, we collected information on the Legend of Dōjōji [8]. user has a large amount of knowledge (items whose knowledge

978-1-7281-7624-6/20/$31.00 ©2020 IEEE 690 DOI 10.1109/CSCI51800.2020.00126 amount information is 9 or 10). In addition, the system determines the amount to be described based on the frequency information. The system omits the items that have already been explained and the items that the user has knowledge of (items with knowledge amount information of 8 or more). Frequency information and interest information are updated each time a description is generated. Frequency information increases the frequency value of the described item by 1. Interest information decrements the value of the described item by 0.1 and increases the value of the non-explained item by 0.1.

III. BACKGROUND SYSTEMS Two systems (a simulation system for Kyōganoko Musume Dōjōji [7,8] and the expansion content generation mechanism [10]) are introduced; these form the basis for the proposed system. A. Simulation System for Kyōganoko Musume Dōjōji Fig. 2. Example of an animation generation In this section, we explain the system that generates an explainable elements in the current scene based on the animation, which is the insertion target of the explanation. Fig.1 explanation target. Thereafter, information on the element to be presents the architecture of a simulation system for Kyōganoko explained is obtained from the knowledge base, based on the Musume Dōjōji. The system receives a command file and explanation type. The mechanism transforms the obtained generates an animation with music. The command file is a file information to generate the case structure of the explanation. Fig. that describes the images to be used, the arrangement and 3 shows an example of knowledge for explanation. animation of those images, the music to be used, and the timing for playing the music. The images are stored in the picture TABLE I. ANALYSES DATA OF “CHŪKEI NO MAI” database for each character and background. The music database stores music generated from the score [11] and existing music. Item Content Continuing from Ranbyōshi, perform a dance at the center Actual stages Fig.2 shows an example of an animation generation. This of the stage, in tune with Noh. animation was generated based on the command file that we Music Creating a music from a score (Nagauta) Kane ni Urami ha kazukazugozaru … (omission) … Lyrics created using the analysis results for Kyōganoko Musume Dōjōji. shinnyo no Tsuki wo nagameakasan Table 1 presents a part of the analysis results [7, 8]. >Characters: Shirabyōshi Hanako (Have Chūkei), Syoke (12 people) >Katarite Images >Musicians: Shamisen (Japanese guitar), Taiko [Drum], Dōjōji Kodutumi [Small Drum], Fue [Flute] Command Display >Place on stage: Main stage, Hanamichi File Animation >Stage equipment: Dōjōji, Kane, Cherry blossoms System IV. EXPLANATION GENERATION This section presents examples of the generation using two Speaker user models. User model 1 (UM1) is a model that has a strong interest in specific items and has a large amount of knowledge. Music Picture User model 2 (UM2) is a model that has a strong interest in many items and a small amount of knowledge. The generation result Database Database for UM1 is presented in Table 2, along with the interest information (Interest) and knowledge amount information Fig. 1. Architecture of the simulation system (Knowledge) for each user model. Fig. 4 shows the classification of the collected data on B. Explanation Content Generation Mechanism Kyōganoko Musume Dōjōji. The value of the node is the value We introduce a mechanism for generating the content of the of interest information and knowledge amount information. We explanation and the data used by the explanation generation have inserted explanations in 14 scenes. Table 3 presents three mechanism. scenes that include an explanation. Fig. 5 presents an example of the explanation of “Shirabyōshi Hanako” in “Chūkei no Mai”. The amount of explanation and the type of explanation are The content of the explanation is as follows: Shirabyōshi determined by the mechanism described in Section 2. The Hanako wanted a memorial service for the bell. The day the bell explanation content generation mechanism receives the was memorialized at Dōjōji, he was the person who came to explanation targets, explanation types, and current scene from perform the dance. The real character of Shirabyōshi Hanako is the explanation control mechanism described in Section 2. The Kiyohime. mechanism selects the element to be explained from the

691 Story: A story written later of the Legend of Anchin and Kiyohime. Next, we present the results for UM2 generation. UM2 is a Shirabyōshi Hanako performs a dance to create a memorial service for the model in which the average value of interest information is bell, but at the end, she reveals her true nature, jumps into the bell, and bigger and the average value of knowledge amount is smaller becomes a snake. than UM1. UM2 is interested in multiple items and has a lot of Character: Anchin / Old monk / Kiyohime / Dōjōji monk knowledge related to the meta information of the work, such as Location: Murogun / Dōjōji / Kumano the history of Dōjōji and the performers. Table 4 shows a part of Object: Kane the explanation provided for UM2. Performance structure: 14 scenes (Michiyuki / (Zenmondō) / Ranbyoushi / Chūkeinomai / Teodori / Mariuta / Hanakasaodori / Kudoki / Yamadukusi / TABLE III. EXAMPLE OF INSERTING EXPLANATIONS INTO SCENES (UM1) Teodori / Shuzudaiko / Kaneiri / Inori / Jatai / Oshimodoshi) → Mainly dance scenes Scene Item Type Amount Dance: Kimusume (ryūkōuta) / Shirabyōshinomai / Shirabyōshi / Marithuki Michiyuki story Unchiku Part (Dance) / Hanakasaodori / Musume [Young girl] / Kakko no odori / Michiyuki character Unique All Suzudaiko / Nohgakari) Michiyuki history_of_the_work Unique All Dance performer: Shirabyōshi Hanako / Shoke Ranbyōshi performance_structure Unchiku Part Music: Jōruri / Shirabyōshi / Noh / Hasiuta (Shohu no eroticism) / Rōdukusi Ranbyōshi dance_performer General All (Yūjo) / Wakitebushi / Koi to sitto no kouta / Yamadukushi / Taueuta Ranbyōshi stage General All (Saotomeodori) Ranbyōshi story Unchiku Part Music performer: (Katarite) / Shamisen [Japanese guitar] / Taiko [Drum] / Chūkeinomai performance_structure Unchiku Part Kodutumi [Small Drum] / Fue [Flute] Chūkeinomai dance General All Music instrument: Shamisen [Japanese guitar] / Taiko [Drum] / Ko / Chūkeinomai music General All Kodutum / Fue [Flute] / Hyōshiita / Suzudaiko / Kakko / Suzu Chūkeinomai character Unique All Stage: Set in Dōjōji Chūkeinomai history_of_the_work Unique All Fig. 3. Example of knowledge for explanation

TABLE II. THE PARAMETERS OF EACH USER MODELS UM1 UM2 Parameter Names Interest Knowledge Interest Knowledge performance_structure 6 10 4 0 dance 3 7 7 3 dance_performer 5 5 5 5 music 4 6 6 4 lyrics 0 10 10 0 music_performer 0 0 10 10 stage 5 5 5 5 story 9 10 1 0 character 7 3 3 7 place 2 8 8 2 object 0 0 10 10 history_in_the_play 0 0 10 10 history_of_the_work 8 2 2 8 konjaku_monogatari 0 8 10 2 Fig. 5. Explanation of the character (Shirabyōshi hanako) dōjōji_engi 0 8 10 2 TABLE IV. EXAMPLE OF INSERTING EXPLANATIONS INTO SCENES (UM2)

The Legend Scene Item Type Amount Kyōganoko of Dōjōji Michiyuki lyrics Unique All Musume Dōjōji 0/9 Michiyuki music_performer Unchiku Part Konjaku Michiyuki place Unique All Monogatari Michiyuki object Unchiku Part Performance History of 0/8 Story structure the work Michiyuki history_in_the_play Unchiku Part 6/10 9/10 8/8 Dōjōji Engi Michiyuki konjaku_monogatari Unique All 0/8 Michiyuki dōjōji_engi Unique All etc. Ranbyōshi dance Unique All Dance Music Stage Unique History in Ranbyōshi music All 3/7 4/8 5/10 Place Object Character the play Ranbyōshi dōjōji_engi Unique All 0/10 0/0 Chūkeinomai performance_structure General All Performer Lyrics Performer 7/9 2/8 Chūkeinomai dance_performer General All 0/10 0/0 5/10 Chūkeinomai lyrics Unique All Chūkeinomai music_performer Unchiku Part Fig. 4. Hierarchical structure of information on Kyōganoko Musume Chūkeinomai stage General All Dōjōji (The values are UM1’s parameters.) Chūkeinomai place Unique All Chūkeinomai object Unchiku Part Chūkeinomai history_in_the_play Unchiku Part Chūkeinomai dance_performer General All

692 Next, we compare two explanation generations using two the explanation generation system. We have prototyped a user models. In the two user models that we prepared, the mechanism for inserting animation and explanatory texts into interest information and the amount of knowledge were set in the generated story. In the present study, we examined the such a way that the parameters were opposite. Fig. 6 presents a method of inserting an explanation into an animation. Then, we graph of how the system provides explanations for each user systematically organized information on the Legend of Dōjōji model. The horizontal axis represents the items to be explained, and examined the explanation strategy of the system. and the vertical axis represents the amount of explanations provided. The solid line represents UM1, and the dashed line ACKNOWLEDGMENT denotes UM2. This research was supported by the Japan Society for the The system provided UM1 with 72 explanations and UM2 Promotion of Science (JSPS KAKENHI), Grant No. 18K18509. with 107 explanations. In addition, the types of items explained REFERENCES in UM1 and the types of items explained in UM2 are opposite in [1] A. Fujii, M. Watanabe, and T. Ishikawa, “Automatic Generation of Term relationship. In particular, the system strongly explained the Descriptions by Web-based Multi-Document Summarization,” IPSJ SIG story, characters, and performance structure in UM1. However, Technical Reports, 2004(1), 31-38, 2004. the system evenly explains each item in UM2, and items with [2] M-A. Clinciu, and H. Hastie, “A survey of explainable AI terminology,” low interest and a large amount of knowledge are hardly Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019), 8-13, explained in UM2. In addition, the type of explanation selected 2019. by the system has characteristics for each user model. The [3] Kyōganoko Musume Dōjōji, “Bandō Tamasaburō butō shū Vol.1 [DVD],” system explained a considerable amount of general knowledge Tokyo: Syotsiku home video, 2003. for UM1 (54% of the total) and provided unique information for [4] M. Kawai, and T. Ogata, “Toward Explanation Generation in a Narrative Generation System,” Proc. of the 37th Annual Meeting of the Japanese UM2 (48% of the total). Cognitive Science Society, P-117. Japan, Gifu, 2020. In this system, the type of explanation depends on the [5] M. Kawai, and T. Ogata, “Analyzing the stage performance structure of Kabuki-dance “Kyōganoko Musume Dōjōji”,” Proc. of the 34th Annual amount of interest. However, the user who is interested in an Conference of the Japanese Society for Artificial Intelligence, 3D1-OS- item does not always have a lot of knowledge about that item. It 22a-04. Japanese Society of Artificial Intelligence, Japan, Tokyo, 2020. is not always the correct answer for the system to present deep [6] T. Akimoto, and T. Ogata, “Towards a Discourse Mechanism in Narrative Generation System: Proposal of a System Introducing Narrative knowledge (unchiku) to users who are strongly interested in an Discourse Theory and Reception Theory,” Cognitive Study, 20(4), 396- item. Therefore, in this study, we have prepared the parameters 420, 2013. of interest and knowledge. If the user was interested in an item [7] M. Kawai, and T. Ogata, “The Stage Structure of a Kabuki-dance: Survey, and the user had knowledge of the item, the unchiku explanation Analysis, and Representation of Kyoganoko Musume Dojoji,” Proc. of the 37th Annual Meeting of the Japanese Cognitive Science Society, P- and the unique explanation were displayed. A future work will 117. Japan, Gifu, 2020. be to examine the effects of interest and knowledge parameters [8] M. Kawai and T. Ogata, “An analysis of Kyōganoko Musume Dōjōji for on the content to be explained and methods used. Thus, we plan the generation of the staging structures of Kabuki from narrative plots,” to experiment with additional user models. SIG-LSE Proc., 63, 25-61, 2019. [9] T. Watanabe, “Musume Dōjōji,” Tokyo: Shinshindo, 1986. V. CONCLUSION [10] J. Ono, T. Ito, and T. Ogata, “Toward Explanation Generation in a Narrative Generation System,” Proc of the 37th Annual Meeting of the The aim of this study was to broaden the approach offered Japanese Cognitive Society, 746-754, 2020. by the expression method in the explanation generation [11] Kineie Yashichi IV, “Nagauta Kyoganoko Musume Dojoji: Shamisen mechanism that we are developing. Thus far, we have developed Bunka Fu,” Hogakusha. 1952. an explanation generation and an animation system, which we have combined; further, we have added a moving illustration to

Fig. 6. Explanations provided for UM1 and UM2

693