Web3d Dance Composer: a Web-Based Ballet Performance Simulation System

Web3D Dance Composer: A Web-based Ballet Performance Simulation System

SOGA Asako Graduate School of Human Informatics, Nagoya University [email protected]

Abstract also allows artists opportunities and possibilities to express their This research approach focuses on sharing 3D animation data feelings. over the Web and creating a system capable of animating virtual ballet performances. Sharing motion data on the Internet allows This paper describes a research framework to develop the anyone to access various artistic dances all over the world. In ballet performance simulation system and shows a dance addition to this, these motion data can be applied in online performance example actually generated by it. systems for educational and artistic purposes. I present Web3D Dance Composer, which enables the creation and editing of classical ballet animation based on VRML. This system allows 2. Sharing and reusing motion data on the Web anyone to easily create and simulate artistic ballet performances as well as realistic stage effects. Since it is important for I developed a simulation system that can easily create ballet classical ballet to provide an artistic harmony from both dance performances on the Web. In this work I focus on a number of choreography and stage&scenario effects, this system can be issues related to sharing and reusing motion data. As there are a useful for teachers and choreographers to simulate their lot of steps to choreograph classical ballet, it is difficult to performance in advance. This system also allows artists collect all steps. The concept underlying my system foresees an opportunities and possibilities to express their feelings. increasing accumulation of these data, and their distributed storage over the network for sharing and reusing by users.

1. Introduction The main issue to share and reuse dance motion data over the Net is describing human motion. For this purpose, I adapted Although many researches on dance using 3-dimensional H-Anim [4], a standard way of representing humanoids in (3D) human animation through motion capture systems have VRML, to a human model of a female ballet dancer [5]. been recently developed, little attention has been given to share I further segmented some motion sequences as basic ballet and use those motion data on the Internet. My research focuses steps. Such independent motion data sequences make it easy to on sharing 3D animation data over the Web and creating a access and recompose actions easily as well as reuse the same system capable of animating virtual ballet performances. data. I also handle characters and motion data independently. For many years, various animations have been composed by This allows us easily to change costumes or characters as well as using applications for choreography [1][2]. Such applications, to change motions. This allows character designers to simulate though widely used until now, do not work on the Web and they these ballet animations with their own characters. are not suitable for online applications. Sharing motion data on Figure 1 shows an image of the sharing and reusing motion the Internet allows anyone to access various artistic dances all data by using this system. Basic steps and solo dances are over the world. In addition to this, these motion data can be increasingly captured and made available on the Net. Users may applied in online systems for educational and artistic purposes. use such data to create their composed steps. Teachers and For example, one can prepare 3D movement catalog, compose choreographers can save their own choreography. On the other pieces by collaborating via the Net, and exhibit pieces on an hand, students and the other users can see and use new electronic dance museum. arrangements of created archives. My goal is to develop and integrate several modules into a system capable of animating realistic virtual ballet performances Solo dances Basic Steps Basic on the Web. I present Web3D Dance Composer, which enables Steps the creation and editing of classical ballet animation based on Virtual Reality Modeling Language (VRML) [3]. This system allows anyone to easily create and simulate artistic ballet Students performances as well as realistic stage effects. Since it is Composed important for classical ballet to provide an artistic harmony from Steps both dance choreography and stage&scenario effects, this system can be useful for teachers and choreographers to simulate their Character performance in advance. Moreover, this system also has other Designers possibilities regarding ballet dance performances. For example, users can change human characters to animals such as dogs and Characters Teachers and Users Choreographers cats, or they can appreciate the performance from any imaginary seat such as a moving seat and a dancer’s viewpoint. This system Figure 1: Sharing and reusing motion data

16 | Papers | ISEA2002 Web3D Dance Composer Motion Database Dance Animation Generation System WWW

Human Archive Registration Model System

Other objects Back Stage Production ground Simulation System

Figure 2: System architecture of WDC

Figure 4: Recording motion data by a professional dancer

Number Examples

Basic Steps 160 Assemblé, Glissade etc. (including Arrangement) (300) (Assemblé Ouvert etc.)

Solo Dances 15 Cupid Variation etc.

Table 1: The number of motion data and examples

Figure 3: User interface of WDC 3.2 System architecture and user interface

Figure 2 indicates the system architecture of WDC. This system consists of 3 subsystems and motion database. The motion data of basic steps should be prepared on the motion 3. Web3D Dance Composer (WDC) database. The other object files such as characters for the human model and background object data can be stored on the Web and users can add more files. 3.1 System overview Figure 3 shows the user interface of WDC. It employs only Web3D Dance Composer (WDC) is a Web-based ballet one window, which consists of a virtual environment based on performance simulation system. Using motion capture data, one VRML and a Java Applet including menus for each subsystem. can easily compose and simulate various ballet performances. All the requirements to run this system are a WWW browser and This system consists of 3 subsystems: a dance animation a VRML Plug-in. This system works on general Windows PCs. generation system, an archive registration system, and a stage production simulation system. 3.3 Motion database The dance animation generation system interactively creates a series of arrangements of a dance from basic ballet steps Each dance has its own style and composing process. previously captured from professional dancers. The archive Especially in the case of classical ballet, there are typical steps registration system registers the dance sequences that may be and methods that have to be considered. Thus, in order to new arrangement of dance archives created by the dance compose a virtual choreography, my research group proposed a animation generation system. Using these registered animations, method for segmenting motion data and describing classical the stage production simulation system creates a dance ballet steps on the computer [6]. In addition to this, we performance according to a scenario. Users can interactively constructed a motion database of basic steps, which were simulate various stage effects such as costumes and background. performed by professional ballet dancers. We have about 160 basic steps and some arrangement steps for each basic step. We As a result from the use of the three subsystems described also have some solo dances for classical ballet. For capturing above, users can finally create classical ballet performances on motion data, we used both an optical motion capture system and the Web. a magnetic system for 5 dancers because it is difficult to use the same systems and dancers to gather all steps. Figure 4 shows a recording scene using the magnetic system. Table 1 shows the number of steps and some examples of our motion database.

ISEA2002 | Papers | 17 Beat 0 1 2 3 4 5 6 7 8

Composed Step1: Piqué Step2: Soussous Step3: Pirouette Steps1

Figure 5: An Example of composed steps

4. Functions of each subsystem

Actor1 Solo Dance1 4.1 Dance animation generation system (1dancer)

Since classical ballet is a dance that typically consists of Actor2 Composed established steps, one can compose ballet choreography by (8dancers / circle) Steps1 connecting basic steps. I constructed a dance animation generation system by using these steps [7]. This system easily Actor3 Composed Composed allows dancers and teachers to generate the arrangement of a (4dancers / line) Steps2 Steps3 dance. Figure 6: An example of scenario The dance animation generation system roughly consists of 3 parts: Step List, Time Line and Control Panel. All names of steps are listed in Step List. Composed steps are shown in the Time Line. In the Control Panel, there are some buttons for the following functions:

(1) Previewing: The user can preview each step at any time before he/she adds it to the timeline.

(2) Speed Control: The system supports to change beat for each step because choreograph is usually depend on the beat like music. It also supports to change tempo to play.

(3) Editing: The system supports to add and delete steps to Figure 7: Characters for the human model edit the list of steps. It also supports to change the turn of steps.

At the beginning, there is a character on the 3D world and there is no animation. To make choreography, the user provides 4.3 Stage production simulation system a sequence of steps in Time Line by selecting steps from the Step List. Each step has some options such as left/right and beat. Using registered animations, the stage production simulation After making a list of steps, one can load the corresponding system creates a dance performance according to a scenario. animation to the character. Once the user loads the animation, Figure 6 shows an example of scenario for a dance performance. he/she can play it anytime. Figure 5 shows an example of The user can post the composed steps or solo dances to the composed steps. To connect some steps, each action can be timeline for each actor. He/she can simulate different animations translated and interpolated. for multi-humans. In order to represent an attractive ballet stage, I prepare the display control system that has the following functions: 4.2 Archive registration system (1) Character: The user can change the character for the The archive registration system is used for registering the human model, some of which are predefined as shown in Figure dance sequences that may be new arrangement of dance archives 7. These characters, as well as any other character on the Web, created by the dance animation generation system. In the present can be easily changed in accordance with users interests. system, 10 of composed steps that have choreographed by a ballet teacher for beginner’s lesson are already registered. The (2) Background: Since the system supports VRML files in user can use these composed steps as well as he/she can which some objects are defined, users can freely adapt the stage appreciate those. This system also registers previously captured characteristics as they like. solo dances in classical ballet and characters for the human model that based on VRML H-Anim. (3) Formation: One can copy the figure of the dancer with the same choreography to simulate more than one of dancers dancing together. In addition to this, it is also possible to change the formation of dancers.

18 | Papers | ISEA2002 Acknowledgement

My Special thanks are due to for providing us with the motion capture studios in Warabi-za Digital Art Factory and School of Design and Architecture, Nagoya City University to capture the classical ballet motions. I also wish to thank to ballet dancers. Among them are Noriko Ogawa (Chieko Sassa Ballet), Kumiko Katoh, Yuko Shimura (Noriko Kobayashi Ballet Theatre), Yurie Shimomura, and Kaori Maruyama (Star Dancers Ballet). I also extend my thanks to Prof. Shigeki Yokoi, Prof. Takami Yasuda, Prof. Bin Umino, and Mr. Kengo Hayashi for their cooperation.

Work in progress and current documents may be found at http://pinakes.soc.toyo.ac.jp/BIN/webdance/ . This Web site also contains the proposed system, Web3D Dance Composer.

Figure 8: Dance Performance Example 1 References

[1] Credo Interactive, Life Forms3.9, 2001, http://www.credo-interactive.com/ [2] Warabi-za Digital Art Factory, 2000, http://www.warabi.or.jp/buyo-fu/ [3] Web3D Consortium, VRML97 Specification, 1997, http://www.web3d.org/Specifications/VRML97/ [4] VRML Humanoid Animation Working Group, The VRML Humanoid Animation specification, 2001, http://www.h-anim.org/Specifications/H-Anim2001/ [5] A. Soga, M. Endo, T. Yasuda, B. Umino and T. Kaiga, "Standardization of Dance Motion Data Using H-anim which are Gathered by Motion Capture Systems and its Application; Construction of a Motion Data Archive for Classical Ballet", Proceedings of the IPSJ-SIGCH Symposium, Japan, 2001, pp.41-48, Japanese. [6] A.Soga, M.Endo, and T.Yasuda, "Motion Description and Composing System for Classic Ballet Animation on the Figure 9: Dance Performance Example 2 Web," Proceedings of 10th IEEE ROMAN, France, 2001, pp.134-139. [7] A. Soga, B. Umino, and T. Yasuda, "A Web-based 5. Dance performance examples Interactive Choreography Simulation System for Ballet", The Journal of the Society for Art and Science, Vol.1, No.1, Figure 8 and Figure 9 show the screen shots of dance 2002, pp.30-38, Japanese. performance examples actually generated by WDC. In the case of Figure 8, all dancers perform the same composed steps on a virtual stage. 8 dancers are arranged in a circle around a dancer. Figure 9 is a little complicated case. 9 dancers perform 3 kinds of composed steps, and the background location is a recreation of “Sakae,” the downtown area in Nagoya City.

6. Conclusion and future works

I presented a Web-based ballet performance simulation system combining a dance animation generation system, an archive registration system, and a stage production simulation system. My proposed system is capable of not only composing choreography, but also simulating scenarios and several stage productions. I verified that this system could successfully simulate some dance performance examples. In addition to this, I could effectively share and reuse motion data on the Web. This allows anyone great opportunities and possibilities to use precious motion data.

In the future work I intend to augment the composing system by supporting functions such as the body part manipulation. I will develop an automatic choreography system for actual ballet lessons.

ISEA2002 | Papers | 19 ��

��

20 | Papers | ISEA2002 �

��

ISEA2002 | Papers | 21