Received January 8th, 2019; Accepted May 27th, 2019 Generating 2.5D Character Animation by Switching the Textures of Rigid Deformation Yuki Morimoto. Atsuko Makita. Takuya Semba. Tokyo Denki University, Kyushu University Tokyo Denki University Tokyo Denki University [email protected] Tokiichiro Takahashi. Tokyo Denki University, ASTRODESIGN Inc. Abstract We generated 2.5D animation from raster images, skeletal animation data, and other data formats. The input image of a character is divided into several parts with arbitrarily assigned joint positions. The joint positions in the motion data and additional points are then applied as the control points of rigid deformation, generating a character animation. Geometric interpolation is replaced by switching of the cell animation images. In experimental evaluations, our animation results were successfully generated without interpolation techniques, a large number of input images, and high editing costs for interpolation. We also introduced a transformation content using the method, and confirmed the entertainment value of the contents. Keywords: 2.5D cartoon model, bone animation, cel animation 1 Introduction has 2D appearance. However the inputs of this method is lim- In the traditional cel animation process, characters are animated ited to simple geometry. Also the method inapplicable to general by making slight changes to each manually drawn frame along bone animations. the time series. However, the process requires more than eight cels per second and is very expensive. To reduce these costs, 2.5D animation of more complicated images can be conducted animators abstract the motions of their characters from a reduced in Live2D Cubism software [3]. The main differences between number of drawn frames in the process called limited animation. the method of Rivers et al. and Cubism are the texture mapping Alternatively, animators reuse parts of the cels such as the mouth, and the morphing parameters of each part. Users of Cubism can eyes, and background. Recently, animation has become a digital correspond the parameter values to the part geometries. The sys- process. However, the overall process in which multiple cels of tem then interpolates the geometries by keyframe animation with each part are drawn and switched to generate the animation has the morphing parameters. The user sequentially edits the ver- not changed [1]. tices of the part geometry at the keyframe. And also the user can edit many vertices at once by deforming the curved surfaces In non-photorealistic rendering (a field of computer graphics), that the vertices are mapped onto. Such detailed editing usually many works have focused on a technique called toon rendering, incurs high costs. Live2D Inc. has released the animation soft- which transforms 3D models to a 2D cartoon or anime-like im- ware Euclid, which improves the viewpoint angle to 360 degree ages. Although toon rendering has recently improved the produc- by switching the textures of each part. For this purpose, Euclid tion cost of animation creation and game development, it encoun- extends the method already employed in Cubism. The switching ters problems when the 2D expressions contradict the 3D world. operation is applied only to parts of face and head. To overcome such problems, some researchers have interpolated between the user-specified 3D geometries viewed from some per- Here, we propose a method that generates character anima- spectives. The interpolation sometimes induces unnatural appear- tions from images, while avoiding unnatural interpolations. Our ances especially when the 3D model is bumpy or complicated. To method replaces interpolation with texture-switching and a rigid avoid this problem, additional editing is needed for interpolation. deformation procedure. Moreover, we correspond part textures viewed from different angles using joints instead of vertices such Rivers et al. [2] proposed a 2.5D cartoon model that generates as in the above 2.5D methods, our method is not reliant on the smooth animations such as 3D animations from one input image texture geometry (which can be uneven and complex). The en- per part. In this method, the user specifies the appearances (ge- tertainment value of the method was assessed in a questionnaire ometry and color) of each part from different viewpoints. The survey of the animation results. Although the resulting anima- result animation is smooth like 3D animation by morphing but it tions are less smooth than other 2.5D methods, the smoothness 16 Received January 8th, 2019; Accepted May 27th, 2019 quality was deemed reasonable by the respondents of our ques- tionnaire survey. 2 Related works 2.1 Rigid deformation Proposed by Alexa et al. [4], the rigid deformation technique reduces the distortion in the deformed geometry. Unlike simple linear interpolation of the vertices, the rigid deformation is an affine transformation excluding rotation and shear as far as possi- ble. Igarashi et al. [5] divided a target image into triangle meshes and specified multiple vertices as control points. Their method enables a fast and smooth deformation with reduced distortion of each triangle. Using similar inputs, Schaefer et al. [6] de- formed an image by affine transforms weighted by the distances between control points and mesh vertices. Further, Jacobson et al. proposed a method that accommodates the positions and rota- Figure 1 Overview. tions of control points, control lines, and control cages within the same framework enabling flexible image deformation [7]. Other 4 Details rigid deformation techniques include the registration of hand- 4.1 Rigid deformation scheme of Schaefer et al. drawn animation [8], image processing methods such as content- We apply rigid deformation proposed by Schaefer et al. using aware image resizing [9]. Although rigid deformation has been moving least-squares method [6]. This method deforms an image extended in various ways, we present the first documented exten- based on the specified positions before and after the deformation, sion of 2.5D animation. while avoiding intuitive distortions. In this way, a wide range of Vertex blending is similar to rigid deformation method proposed motions are covered in one image. The method maps the input by Schaefer et al [6]. Vertex blending is generally used for cal- image onto flat triangle meshes, and calculates the positions of culating the vertex positions of a 3D model in bone animation. the vertices on the meshes. Vertex blending operates by summing the weighted affine trans- formations of bone joints. Rigid deformation in the method pro- ~f (v) f (v) = jv − p∗j + q∗; ~f (v) = ∑qˆ A ~ i i posed by Schaefer et al. is limited to translation, rotation, and j f (v)j i ! ! uniform scaling of both the x and y axes. These results are less T pˆ v − p∗ 1 distorted in human perception. A = w i ; w = ; i i − ? − − ? i j − j4 (1) pˆi (v p∗) pi v 2.2 Combination of 2D and 3D expressions pˆi = pi − p∗; qˆi = qi − q∗; Some research papers have simultaneously captured the textures ∑i wi pi ∑i wiqi p∗ = ; q∗ = of 2D expressions with the smoothness of 3D animations [10, 11, ∑i wi ∑i wi 12]. These methods map 2D textures onto 3D models. Based on Here, v is the vertex of the lattice divided in the input image, similar concepts, other researchers have arranged 2D layers into p and q are the control points before and after deformation, re- 3D spaces [13, 14], but these methods have unique goals. spectively, and f (v) is the deformation function applied to v. The weight function w is related to the distance between each 3 Our method vertex and control-point, and m is the number of vertices when 3.1 Overview (i = 0;1;:::;m). Please refer [6] for more details. The overview of our system is described below, with reference to Fig. 1. First, the character image is segmented into parts (Fig. 4.2 Two kinds of control points 1(a)), which are the input for our system. Here, the green trian- 4.2.1 Joint-based control points gles and green arrows indicate the joint positions and bones of the Our method defines control points as the joint positions of each character, respectively. Our system requires two or more different part. The joint positions are specified by a mouse click on the part images of the angles of each part. The body parts are shown by images. The required positions after deformation are obtained by the rectangles in Fig. 1(a). The body-part images that are inputs scaling the motion data to the size of the character. Note that rigid for our system are corresponded to their orientations. Hence, our joint deformation of the long parts, such as arms and legs, would system selects the image with the closest direction of each part yield round shapes. To avoid this problem, the control points (see Fig. 1(b)). The additional control points can be specified for in our method are arranged along the bones. Here i is the joint aligning arbitrary positions of different parts (Fig. 1 (c)). In this number, j is the neighboring joint of i, and k = 1;2;:::;n, where example, the rigid deformation will align the blue and red posi- n is the number of control points between joints i and j. The joint joint tions shown in Fig. 1(c). Then, bone animation is applied. As position Pi; j;1 of joint i corresponds to Pi . To additionally shown in Fig. 1(d), the part images are switched and deformed constrain the rigid deformation, we add the sub-joints as control to correspond to the direction or angle of the joints of the input points approximately every four pixels between the joints, fol- skeletal animation. lowing a parent-child relation. These sub-control points after de- 17 Received January 8th, 2019; Accepted May 27th, 2019 joint Here, the function H transforms the 3D positions to 2D positions formation Qi; j;k are arranged on the bones between joints i and j at the interval of jQ joint − Q joint j=n (shown as black triangles in on the x-y plane, and j is the parent joint of i.