Journal of Software

Streamlined Workflow for 3D Modeling with Animated Characters

Wei-Chieh Chang, Wei-Min Jeng* Department of Computer Science and Information Management, Soochow University, Taipei, Taiwan.

* Corresponding author. Tel.: 886-2-2311-1531 ext. 3811; email: [email protected] Manuscript submitted December 14, 2014; accepted June 17, 2015. doi: 10.17706/jsw.10.7.912-918

Abstract: With the proliferation of computer graphics and multimedia technology, there are abundant choices of three-dimensional (3D) modeling and character animation professional software in the market. However, authoring using different software products collectively still remains a difficult task considering the distinct features of proprietary software artifacts. The main purpose of this paper is to propose a streamlined workflow aiming at performing the 3D modeling construction with animated characters. As the concept of Wikinomics emerges, open source based software and products are utilized to provide the maximum collective throughput potential. The proposed SketchUp and MikumikuDance workflow (SUMD) introduces the novel method for plentiful of potential applications in light of the augmented capabilities of the integrated process. The goal is to assist professional users associated with different domains and sustain their high enthusiasm levels during the modeling process. Rather than manipulating more than one software products with incompatible properties, the workflow users will be guided through the challenging process to complete their 3D construction with animated characters in the proposed SUMD workflow to create potentials for more future application use.

Key words: Augmented reality, SketchUp, animation, workflow, MikuMikuDance.

1. Introduction Augmented reality is a live direct or indirect view of a physical, real-world environment whose elements are augmented or supplemented by computer-generated sensory input such as sound, video, or graphics [1], [2]. With the advanced augmented reality technology like computer vision and object recognition, the emerging computer-based tools can help users interact and digitally manipulate with various aspects of the real world surrounding. The idea of augmented reality has been applied in many fields since its introduction in the 90’s. Many of the created virtual objects can be overlaid with the real world scenes in various settings to exhibit special effects by using augmented reality technology. For 3D modeling, the designer would appreciate the idea if the automated tool can assist in the process by offering the capability of both the 3D scene together with the animated characters to attain the best possible effects. Take relatively popular 3D Modeling computer software SketchUp for examples, SketchUp is always used in 3D visualization by erecting buildings with CAD programs or integrating construction locations with Google Earth. Conversely, MikuMikuDance (MMD) is mainly applied in 3D character animation. It is often used by animating static characters with Microsoft Kinect sensor to create special effects forlater display. In order to design the workflow efficiently while keeping both benefits from the originally different software, the key challenge is to streamline all the operations for typical users in maximizing their productivity. The proposed

912 Volume 10, Number 7, July 2015 Journal of Software

SUMD consists of two parts, namely SketchUp and MMD operation, to seamlessly integrate the two operations without challenging user intervention. An integrated 3D modeling with animated characters has many appealing elements, such as beautiful backgrounds and rhythmical dancing. The majority of professionals like to share their finished models for their clients or even for public download. Our proposed SUMD workflow offers the capability for the professionals or even the novices a streamlined process to enrich the 3D scene object with blended user-defined animated characters. The SUMD workflow successfully demonstrates better manipulation capability in immersing of motion characters into static scene smoothly to enable more potential applications than current methods such as commonly used 3DMax software one.

2. 3D Modeling and Character Animation 3D Modeling tool plays a key role in many professional applications such as computer-aided interior design. The representative 3D modeling software, Google SketchUp [3], aims to conduct 3D scene construction from scratch with its rich functionalities. In spite of the fact that most of prior uses of SketchUp have been focused on 3D visualization only, it becomes more popular for AutoCAD users to design construction precisely under AutoCAD’s 2D setting and later render the results in 3D scene [4]. This method provides 3D visualization of real scenes easily from 2D drafting. Integrating Google Earth with SketchUp is another way to create photorealistic 3D construction by utilizing Google Earth photos. Streamlined structuring operation with geometry is what SketchUp emphasizes. It is another feature of SketchUp to make modeling operation easily for novices to learn. SketchUp defines digital lines to generate various surfaces which are combined with sharp, forms and objects. Fig. 1 exhibits the finished photorealistic 3D SketchUp model of an interior design layout. In spite of the rich 2D/3D architectural layout design features, SketchUp is not specially made for character animation purpose like other software such as 3DMax. As a matter of fact, the 3D modeling software is still difficult for average users to master the entire 3D scene construction process in which it is required to accommodate animated characters with the sole layout-based software packages in short amount of time.

Fig. 1. 3D model construction (source: 3D Warehouse SketchUp website).

MMD has been the free 3D animation software publicly available since 2010 [5]. Traditional way for rigging characters is to conduct skeleton bones and then rig the skeleton with character construction. Due to different models of characters, the skeleton only fits to its original models and it cannot reuse in other ones. Users may gather pose data together to develop motion data for animation. Users have to conduct the tedious process repeatedly until rigged character is completed. Fortunately, MMD improves the rigging

913 Volume 10, Number 7, July 2015 Journal of Software

process effectively and increases augmented capability of character models by using PMDEditor. Another way for animating rigged characters is to utilize an external camera device such as Microsoft Kinectto convert real movements into motion data [6]. MMD can animate rigged characters easily by using Kinect plugin that supports motion data. The environment variation can be detected by using Kinect programs which can capture contrasting picture difference to develop motion data and integrate it into MMD for use. The rigged character named Miku Hatsune is an internal character that has been rigged MMD skeletons completely. Real people movements can be detected by using Kinect plugin and shown on upper right screen with a human icon. The rigged character can be animated automatically like the human icon does the same motion when the real person actives. It is instinctive for users to pose and check motions directly to develop MMD motion data. Most MMD uses have been focused on interactive animation by exploiting pedagogical practices for strong bodily-kinesthetic intelligence [7]. PMDEditor is the external software for MMD to rig skeletons with static characters and check construction information in detail. In order to provide an augmented reality look and feel for aesthetic appreciation, demand from professional users for a streamlined workflow is high with the explosive growth of user-generated content [8]. Since involving programs of any workflow more likely are provided by different software vendors that are intrinsically hard-to-integrate in terms of format interchangeability accompanied by the building block components, the most critical factor to the success of resulting product of workflow depends highly on the seamless integration between the heterogeneous programs.

3. The SUMD Workflow

Fig. 2. SUMD workflow.

According to the technology acceptance model (TAM) [9], perceived usefulness and perceived ease-of-use are the two united reasons for users to accept new technology. Perceived usefulness is defined as the degree to which a person believes that using a particular system would enhance his or her job performance. In this study, usefulness can be thought as the degree to which users believe that using the particular 3D modeling software would enhance the quality of performances and increase enjoyment levels. Likewise, perceived ease-of-use is defined as the degree to which a person believes that using a custom made system would result in less efforts. The professionals who are good at modifying function or modeling amazing artifacts can share their performances without infringing intellectual property rights. The finished models are very popular among users because novices almost lack experiences in actual drill. To make beginners learn more easily, the workflow could simplify process operation by exploiting finished models. Thus, the goal of our

914 Volume 10, Number 7, July 2015 Journal of Software

proposed SUMD is to integrate SketchUp 3D construction with MMD animated characters in a streamlined fashion to perform productive operations for professionals. Although finished models are convenient for users, integrating different formats with distinct software still remains a major problem. Common software almost have fixed format which can’t be used compatibly by crossing different platforms. The major technical issues involved in the overall process include language system differences, format conversion, and coordinate system etc. Fig. 2 illustrates the entire workflow of the proposed SUMD workflow that consists of mainly SketchUp and MMD components. Because of the different language systems used by potential users, MMD has to be properly configured to overcome the language discrepancies. Microsoft AppLocale can be used to convert Japanese into other symbols. Downloading finished 3D models is the first thing of the workflow. Users can download MMD for free from Vocaloid Promotion Video Project (VPVP) website. The MMD program should be saved in zip format so that can be dealt with language symbols afterwards. The most important issue in this step is that all downloaded file contents with Japanese names must be renamed in English. The reason is that zip files with Japanese may have system errors during self-extraction. After self-extracting process is done, the MMD program can be converted from a self-extraction file into a normal system file by using AppLocale piaip. For example, the users can download the Florence Duomo of SketchUp construction from open source Google 3D Warehouse with designated format. On the other hand, rigged characters and motion data can be downloaded from BowlRoll with appropriate bone names which have been set beforehand in different skeleton methods. The 3D construction then can be imported for SketchUp use and then modified by using SketchUp toolkits to fit the background size for later use. Because of file incompatibility problem between SketchUp and MMD formats, the modified 3D construction should be converted from SketchUp into DirectX format before exporting. To make users convert models more easily, SketchUp plugin is the best choice in the process. Format conversion can be done effortlessly with parameter control and its coordinate errors can be decreased during conversion. By using the SUMD operation, the 3DRad SketchUp plugin can be used to convert from SketchUp construction format into other formats. The way how 3DRad plugin is installed is in fact similar to the template setting operation after putting the 3DRad ruby files into SketchUp plugins folder. One of the commonly seen problems during the 3D modeling workflow operations such as the reported coordinate errorsin exported finished background could be discovered by the above-mentioned PMDEditor toolkit. System notification windows indicate that states of the format conversion; the converted file then can be imported into MMD as the animation background. 3.1. Character AnimationWith MMD Under the proposed SUMD workflow, what users have to do is to export rigged characters into MMD and then to load motion data into the rigged characters. At the end of operation, the rigged characters could animate manipulated by users. Animated characters can be performed under MMD in three steps. The details are described as follows. First, rigged characters must be imported into MMD. Motion data then can be put into rigged characters. The skeleton bones of rigged characters are selected automatically when users import the England character into MMD. Creations of simple animated characters are finished after integrating motion data with rigged characters. Finally, animated characters must be checked in MMD preview. Animated characters can function normally when the skeleton bone names of rigged characters are similar to motion data ones. Motion data can be modified by using record points of frame manipulation. The change of record points could affect interpolation curve directly. SketchUp backgrounds in DirectX format can be imported into MMD when animated characters are completed. Fig. 3 shows that the Florence Duomo background can be imported into MMD withthe accessory panel operations. The background locationalso can be modified freely by inputting its centering geometric coordinates. As a result, each imported 3D construction object created in the early stages of the SUMD workflow is treated as an independent entity

915 Volume 10, Number 7, July 2015 Journal of Software

and its designated location can be rearranged at later time according to the need of the whole setting.

Fig. 3. 3D background integrated with animated character (source: Google 3D Warehouse).

3.2. Immersion of Motion Characters One of the technical edges SUMD has over other competing software is the ability to seamlessly immerse the user’s animated characters in the static scene. Traditional immersion accomplishes the same task by blending the scene with motion characters by repeatedly setting the lighting, scaling, and texture mapping parameters during the importing process. It takes experienced users to handle the material editor in order to exhibit the realistic appearance of the integrated 3D output. On the contrary, SUMD offers built-in features to easily import the pre-processed Kinect motion characters from cloud-based downloadable 3D model website or user-created SketchUp one. In addition to the traditional integrated animation use, SUMD can also be applied in other disciplines such as interior design application. Users can download the interior design office developed by Manojkumar from Google 3D Warehouse. The interior design office is constructed by integrating AutoCAD drafting with SketchUp. As done in previous operation of SUMD, the interior design office in SketchUp format can be modified directly and then exported to MMD by using 3DRad plugin. In addition to the dynamic motion data of animated characters and animation camera, MMD also has static type called the character pose data. The pose data can be developed by using Kinect plugin or handmade manipulation. Fig. 4 illustrates the animated characters immersions into a static interior scene with required functionalities including hidden line removal. The superior feature facilitates the potential new uses of the proposed workflow in new applications which requires the presence of motion characters in 3D constructions or modeling. Contrasting to the old way of blue print type of presentation used by most interior design professionals, the users now can easily get an augmented reality type of first-hand look-and-feel by immersing himself in the surroundings. It definitely provides a better communication tool without building a show-case prototype. In

916 Volume 10, Number 7, July 2015 Journal of Software

addition to the user friendly rendering advantage provided by the immersion process, the immersed motion characters are modifiable with provided interface and the finished outputs are reusable for other occasions to maximize the proposed workflow benefits.

Fig. 4. Interior scene with imported kinect animated characters. (source: 3D Warehouse cloud service).

4. Conclusion 3D modeling and character animation are frequently used applications nowadays in a wide variety of fields. Despite its overwhelming technical advances, the majority of traditional 3D modeling software enjoys more success in rendering non-human character animation. Most of the current software suffers from two drawbacks including discontinued-support and complex operating details. With the proliferation of commonly accepted Wikinomics concepts, users who lack experiences in constructing 3D buildings and characters now may download finished models from open resource materials. On the other hand, with the thought of Perceived Usefulness and Ease-of-Use borrowed from TAM model, our proposed SUMD workflow aims to make the operation easier and effective during the integration phase. Both SketchUp and MikuMikuDance were used to build the transparent workflow by mitigating the technical problems including format conversions and language differences. To augment the computer-generated scene with better effects such as human character immersion, animation could complement the deficiencies if the two products can work collectively in a productive way for their users. The catch is to resolve the intrinsic differences between heterogeneous products to facilitate its use. Aiming at providing the streamlined workflow for creating 3D model construction with animated characters, the proposed SUMD resolves the incompatibility between proprietary software and allows the users build their own artifacts without knowing the technical details. Another major feature of the workflow is the built-in capability of immersion of motion characters into the 3D environment. The potential users include interior design architects and their clients who can both communicate with the help of the proposed software tool. The additional animated characters augment the existing 3D modeling software tool by offering the direct look-and-fill of the blended effects. By offering the comprehensive workflow that successfully integrate the two major software products with their distinct features, professionals from different domains are guided through the process to build the 3D model easily and sustain their high enthusiasm level through SUMD. Both prominent SketchUp and

917 Volume 10, Number 7, July 2015 Journal of Software

MMD software are integrated seamlessly in the proposed workflow and the built-in authoring environment maintains high quality of theme backgrounds and rhythmical movements to fulfill the various professional application requirements from different disciplines. Future work could include developing portable building block-based components and on-line interactive operating environment for a broader range of potential uses.

References [1] Azuma, Ronald T. (1997). A Survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355-385. [2] Talmaki, S., Dong, S. & Kamat, V. (2010). Geospatial databases and augmented reality visualization for improving safety in urban excavation. Construction Research Congress, 91-101. [3] Hong, D., Yang, G., & Wang, Z. (2008). Construction of virtual campus based on arcscene and sketchup. J. Computed Technology and Development, 12, 14. [4] Khiati, S. (2011). Acad and 3D visualization software in design education: Is one package enough. Journal of Engineering & Applied Science, 3(2), 91-100. [5] Liu, J., & Grant, S. (2012). Avatar-an online voice chat robot with 3D avatar and artificial intelligence. Proceedings of the 2012 Midwest Section Conference of the American Society for Engineering Education. [6] Hsu, H. (2011). The potential of kinect as interactive educational technology. Proceedings of the 2011 2nd International Conference on Education and Management Technology. [7] John, P., & Sutherland, R. (2005). Affordance, opportunity and the pedagogical implications of ICT. Educational Review, 57(4), 405-413. [8] Balasubramaniam, N. (2009). User-generated content. Proceedings of Business Aspects of the Internet of Things, Seminar of Advanced Topic. [9] Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results, Ph.D. Thesis, Massachusetts Institute of Technology.

Wei-Chieh Chang received her B.S. degree in information management from Tamkang University of Taipei, Taiwan and expect to receive the M.S. in computer science and information management from Soochow University of Taipei, Taiwan. She is specialized in software programming and entertainment multimedia design. Her current research interests include 3D modeling and character animation in which software interface and compatibilityis an issue. Ms. Chang lately received the microsoft technology associate (MTA) and adobe photoshop CS5 certification. She plans to look for a job in information security and computer programming after her graduation.

Wei-Min Jeng received his B.S. degree in management science from National Chiao-Tung University, Taiwan, the M.S. degree in computer science from University of Texas at Arlington, and the Ph.D. degree in computer science from University of Houston. He is an associate professor of the Department of Computer Science and Information Management of Soochow University, Taipei, Taiwan. He was with Decision Consultant Inc., Dallas, Texas, from 1989 to 1990. He later joined Positron Corporation, Houston, Texas, from 1990 to 1993. His areas of research include medical imaging modality and high performance computing.

918 Volume 10, Number 7, July 2015