Eheritage of Shadow Puppetry: Creation and Manipulation
Total Page:16
File Type:pdf, Size:1020Kb
eHeritage of Shadow Puppetry: Creation and Manipulation Min Lin∗1, Zhenzhen Hu∗2 , Si Liu1 , Meng Wang2, Richang Hong2, Shuicheng Yan1 1ECE Department, National University of Singapore 2School of Computer and Information, Hefei University of Technology {mavenlin, huzhen.ice, eric.mengwang, hongrc.hfut}@gmail.com, {dcslius, eleyans}@nus.edu.sg ABSTRACT children and adults, and is also popular in many other coun- 1 To preserve the precious traditional heritage Chinese shadow tries around the world . We focus on Chinese shadow pup- puppetry, we propose the puppetry eHeritage, including a petry in this work. Chinese shadow puppetry, as shown creator module and a manipulator module. The creator in Figure 1(a), is a traditional artistic form of theater per- module accepts a frontal view face image and a profile face formance with colorful silhouette figures. These figures are produced with puppets made of leather or paper. A tradi- image of the user as input, and automatically generates the 2 corresponding puppet, which looks like the original person tional Chinese shadow puppet is composed of a head part and meanwhile has some typical characteristics of traditional shown in Figure 1(b) and a body part shown in Figure 1(c). Chinese shadow puppetry. In order to create the puppet, we As an ancient form of storytelling, sticks and flat puppets first extract the central profile curve and warp the reference are manipulated behind an illuminated background to cre- puppet eye and eyebrow to the shape of the frontal view ate moving pictures [1]. By moving both the puppets and eye and eyebrow. Then we transfer the puppet texture to the light source, various effects can be achieved. But in the the real face area. The manipulator module can accept the 21st century, shadow puppetry in China is on a steep and script provided by the user as input and automatically gener- fast decline. Audiences and apprentices are evaporating at ate the motion sequences. Technically, we first learn atomic an alarming rate. To protect this ancient artistic heritage, motions from a set of shadow puppetry videos. A scripting China's State Council put Chinese shadow puppetry in the system converts the user's input to atomic motions, and fi- first list of National Intangible Cultural Heritage of China in nally synthesizes the animation based on the atomic motion 2006, and United Nations Educational, Scientific and Cul- tural Organization (UNESCO) listed the artistic form in In- instances. For better visual effects, we propose the sparsity 3 optimization over simplexes formulation to automatically as- tangible Cultural Heritage in 2011 . semble weighted instances of different atomic actions into a Nowadays, the preservation of culture heritage attracts smooth shadow puppetry animation sequence. We evaluate growing attentions of the world. Our artistic motivation is the performance of the creator module and the manipula- to develop a system by applying the latest multimedia tech- tor module sequentially. Extensive experimental results on nologies to aid preservation, interpretation, and dissemina- the creation of puppetry characters and puppetry plays well tion of this ancient cultural heritage. To attract more people demonstrate the effectiveness of the proposed system. to be interested in shadow puppetry, we design two puppetry modules, including a creator module and a manipulator Categories and Subject Descriptors module that people all over the world can experience the fun J.5 [Arts and Humanities]: Arts, fine and performing of the puppet creating and performing by themselves. The input of the creator module is two face images. One Keywords of the most important advantages of our creator module is personalization. The making process of a puppet includes Chinese shadow puppetry; face rendering; sparsity optimiza- 4 tion over simplex; animation seven steps which are all complex and ingenious . Our creator module can automatically generate anyone's person- 1. INTRODUCTION alized puppet, and the process is almost immediate. We pay Shadow puppetry has a long history in China, Indone- special attention to keeping the characteristics of the tra- sia, India, Greece, etc. as a form of entertainment for both ditional puppet during the automatic creator process. For example, the puppet's face has long narrow eyes, a small ∗indicates equal contribution mouth and a straight bridge of nose as shown in Figure 1(b). Technically, we extract the central profile curve from the profile face image and warp the reference puppet eye into Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- 1http://en.wikipedia.org/wiki/Shadow puppetry tion on the first page. Copyrights for components of this work owned by others than 2In this paper, we make puppet and puppetry interchange- ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- able. publish, to post on servers or to redistribute to lists, requires prior specific permission 3 and/or a fee. Request permissions from [email protected]. http://www.unesco.org/culture/ich/index.php? lg=en&pg=00011&RL=00421 MM’13, October 21–25, 2013, Barcelona, Spain. 4 Copyright 2013 ACM 978-1-4503-2404-5/13/10 ...$15.00. http://www.travelchinaguide.com/intro/focus/shadow- http://dx.doi.org/10.1145/2502081.2502104. puppetry.htm Area Chair: Marc Cavazza 183 (d) (e) Module I: Puppetry creator (a) # Script Puppet.sit(duration=20) walk …… sit bow Puppet.walk(distance=10, … … … … duration=15) (b) …… Puppet.bow(duration=20) …… (c) Shadow puppetry culture (f) Module II: Puppetry manipulator Figure 1: (a) A scene of traiditonal Chinese shadow puppetry. When light penetrates through a translucent sheet of cloth, the audience will see the \shadows", silhouettes in profile. (b) Faces of puppets are characterized by exquisite headdresses and representative facial features, such as thin eyebrows. (c) The puppets are manipulated with sticks through three keypoints, i.e., neck, left hand and right hand. (d) and (e) are two results of our puppetry creator module for male and female respectively. (f) is an exemplar sequence generated based on user-provided scripts by the puppetry manipulator module. All of the figures in this paper are best viewed in original PDF file. the frontal view eye simultaneously. Then we transfer the literature is mainly divided into two categories: tangible puppet texture as the texture of the profile curve. One male cultural heritages and intangible cultural heritages. Tangi- and one female puppet generated by our creator module are ble cultural heritages5 include buildings and historic places, shown in Figure 1(d) and Figure 1(e). monuments, artifacts, etc. Anna Paviotti et al. [5] dealt We also design a manipulator module. For this module, we with the problem of estimating the lighting field of the mul- aim to preserve the performance pattern of Chinese shadow tispectral acquisition of frescoes by a variational method. puppetry during the shadow puppetry manipulation process. Lior Wolf et al. [6] studied the task of finding the joins of There are several basic puppet motion patterns (denoted as the Cairo Genizah which is a precious collection of mainly atomic actions afterwards), such as walk, dance, fight, nod, Jewish texts. Tao Luo et al. [7] presented a multi-scale laugh, etc. Besides, as shown in Figure 1(c), the real puppet framework to generate 3D line drawing for archaeological il- is controlled with three sticks which are fixed on the pup- lustration. Intangible culture heritage6 includes traditional pet's neck and two hands separately, and the motion pattern festivals, oral traditions, oral epics, customs, ways of life, of other puppet parts is affected by gravity. Therefore, we traditional crafts, etc. Markus Seidl et al. [8] proposed a need to simulate all the atomic actions in the puppet style. detection of gradual transitions in historic material. Anu- There are some existing research works on shadow puppets pama Mallik et al. [9] studied on the preservation of Indian focusing on the user's body interaction with the virtual pup- classical dance. The shadow puppetry belongs to intangible pet, i.e., the puppet's motion imitates the user's motion cultural heritages. [2],[3],[4]. But all of them cannot preserve the puppet's spe- 2.2 Face Rendering cific motion style. To the contrary, our module can convey Digital image processing provides a solid foundation for this traditional artistic charm completely. Our manipulator building artistic rendering algorithms. All image-based artis- module can directly accept text scripts as input and display tic rendering approaches utilize image processing operations the specified motions accordingly. The interface is shown in in some forms to extract information or synthesize results Figure 1(f). For manipulation, we identify atomic motions [10]. Non-photorealistic rendering focuses on enabling a for the animation and collect instances from a set of shadow wide variety of expressive styles of digital arts. Among these puppetry videos. A scripting system converts the user input arts, cartoon and paper cutting are the most related to our into atomic motions, and finally synthesizes the animation work. using the collected instances. Cartoon: A cartoon is a form of two-dimensional il- The organization of the rest of this paper is as follows. lustrated visual art with a typically non-realistic or semi- In Section 2, we provide a review of related work. Section realistic drawing or painting. CharToon system [11] pro- 3 makes an overview of our puppetry module: creator and vides special skeleton-driven components, an extensive set manipulator. Next, in Section 4 and 5, more detailed step of building blocks to design faces and the support to re- by step introduction of the modules is presented.